Drone-based vr/ar device recharging system

ABSTRACT

Disclosed is a system for drone-based recharging of a VR/AR wearable assembly. A system is disclosed comprising a wireless charging device; a wearable assembly; a light field capture VR/AR device comprising a first charging surface and a second connective surface, the first charging surface configured to be positioned on the wireless charging device, the second connective surface configured to be communicatively coupled to the wearable assembly; and a drone device including a network interface, the drone device configured to: receive notifications from the light field capture VR/AR device, the notification indicating that the light field capture VR/AR device is fully charged, remove the light field capture VR/AR device from the charging device, and attach the light field capture VR/AR device to the wearable assembly.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Utility Patent No.62/451,656, entitled “Headphone Based Modular VR/AR Platform,” filed onJan. 27, 2017, U.S. Provisional Patent No. 62/454,716, entitled“Headphone Based Modular VR/AR Platform,” filed on Feb. 3, 2017, U.S.Provisional Patent No. 62/620,082, entitled “Headphone Based ModularVR/AR Platform,” filed on Jan. 22, 2018, and U.S. application Ser. No.15/877,569, entitled “Headphone Based Modular VR/AR Platform,” filed onJan. 23, 2018, each application hereby incorporated by reference for allpurposes.

COPYRIGHT NOTICE

This application includes material that may be subject to copyrightprotection. The copyright owner has no objection to the facsimilereproduction by anyone of the patent disclosure, as it appears in thePatent and Trademark Office files or records, but otherwise reserves allcopyright rights whatsoever.

BACKGROUND

The disclosure relates to virtual reality (“VR”) or augmented reality(“AR”) devices and, in particular, to a headphone-based modular VR/ARplatform.

VR and AR systems offer immersive and involving experiences that place auser in new worlds previously unimagined. Present technology is directedto wearable devices, such as goggles or face-mounted devices that canretain cellular phones or other imaging devices to project stereoscopicimages viewable by a user wearing the goggles or phone-containing mount.

Generally, a VR or AR device includes some or all of the followingsubsystems.

First, VR or AR devices include a display subsystem which generallyincludes one or more display devices mounted near a user's eyes as aface mask or goggles. For example, the OCULUS RIFT® CV1 includes twoOLED displays with a combined resolution of 2160×1200 pixels and a 90 Hzrefresh rate. Generally, these displays are designed to project VRand/or AR scenes to a user. Adjacent to these displays are adjustablelenses designed to alter the projection of the display devices.

Second, some VR or AR devices include a head tracking subsysteminstalled in the front portion of the device (i.e., the portionincluding the screen) designed to monitor the position of a user's headwhile wearing the VR or AR device. Common head tracking subsystemsinclude accelerometers, gyroscopes, and magnetometers. The head trackingsubsystem transmits information regarding the position of a user's head(to a tethered or mobile device placed in the VR or AR device) to enablethe display to be updated and thus simulate a user “looking around” athree-dimensional VR or AR space.

Third, some VR or AR devices include a positional tracking subsystemdesigned to monitor the user's position within a three-dimensionalspace. In general, these systems record the user's position and transmitpositional information to enable the display device to update based onthe user's calculated position within a three-dimensional space. Varioustechniques have been implemented for providing positional tracking. In afirst implementation, a VR or AR device is equipped with numerousinfrared (“IR”) light emitting diodes (“LEDs”). These IR LEDs emitinfrared light which is tracked by one or more mounted cameras whichtranslate the movement of the IR LED light to a three-dimensionalcoordinate representing the user's location (and thus movement) througha three-dimensional space. In a second implementation, a VR or AR deviceis equipped with numerous photosensors designed to detect light emittedfrom fixed light projection devices distributed in a space around theuser. In this system, the projection devices enable the VR or AR deviceto detect its orientation using the projection devices as fixedreference points.

While some VR or AR devices including the subsystems discussed above arecapable of providing an immersive VR/AR experience, they suffer fromnumerous deficiencies. Just a few such deficiencies are identifiedbelow.

As more functionality needs to be included in such devices they becomeheavy and cumbersome and thus limited by how much can practicably beincorporated into such an apparatus that is worn on the face of a user.Such limitations as size, weight and battery life significantly impactthe quality of experience a user can take away from goggle-based VR orAR experiences.

Current VR or AR devices are highly integrated. That is, the subsystemsdiscussed above are designed to work as a single, monolithic unit. Forexample, photosensors or IR LEDs are integrated throughout the VR or ARdevice (e.g., on the outside of the display portion, in the harness,etc.). Thus, if a user wishes to upgrade portions of the VR or ARdevice, the user is required to replace the entire VR or AR device sincethe entire device is designed to work as an interdependent whole (likean obsolete cell phone for example). Additionally, current VR or ARdevices are limited in functionality based on the components within theVR or AR device itself. Thus, users are limited in functionality thatcan be performed by the VR or AR devices.

Many current VR or AR devices are primarily designed to enable a user toview and interact with and within a three-dimensional scene. Generally,to generate scenes for use with a VR or AR device, developers arerequired to generate three-dimensional scenes using external equipment.For example, developers may generate virtual three-dimensional scenesusing three-dimensional rendering software or may generate virtualrepresentations of physical spaces using numerous cameras and lightsources.

Mobile VR or AR devices (i.e., untethered VR or AR devices) are limitedin battery life due to the demands placed on batteries powering the VRor AR device. Tethered VR or AR devices may provide unlimited power viaa physical connection, but necessarily limit the mobility of the VR orAR device due to the tether. Conversely, mobile VR or AR devices allowfor unrestrained movement of the user, but are necessarily limited inbattery life due to the use of limited batteries.

BRIEF SUMMARY

The present disclosure describes a flexible platform based around aheadphone form factor that permits modular connectivity of VR and ARsupport devices, such as positioning components, light field components,audio and video receptors, projectors, modular processor connectivity,novel charging technology, input devices, haptic components, andflexible connectivity options to provide a physical platform upon whichhigh function VR and AR applications and experiences can be modularlybuilt.

In alternative art, headphones have been a ubiquitous presence in theaudio field for many years and ear encompassing headphones have beenutilized by music aficionados and casual listeners since the 1960's andprior. Presently, while ear buds and small form factor earphones havebecome popular, full size ear encompassing headphones have also made asignificant comeback and are now well accepted.

In embodiments disclosed herein, the earphones or headphones (as theseterms are used interchangeably herein), and/or the removable andinterchangeable “pucks” defining some or all functionality for theheadphone assembly that will be described further herein, can compriseone or more of the following components: spherical lightfield/reflectance field image capturing elements; audio and videopickups; micro projector(s); an OLED video display (with or withouttouchscreen capability); trackball, toggle switch and/or hard or softkeypad input devices; positioning sensors to permit high resolutionpositioning of a user within the VR or AR environment; head trackingsensors; eye tracking systems; goggle mounts for receiving andintercommunicating with VR/AR goggles of differing capability; dronemanagement capabilities; modular processor and input/output connectors;and a battery charging platform for extended use, comprising eitherlarger physical batteries (since a headphone is more easily configuredto handle a large size battery), or a rechargeable platform wherebybattery carrying drone devices can interact with the headphone platformto provide continuous charging power to the device.

The flexible VR/AR platform of the present disclosure is designed to bea modular assembly, such that a wearer/user can flexibly interchangecomponents on the headphone assembly. As disclosed herein, the headphoneassembly is designed to incorporate a removable mechanically andelectrically coupleable “puck” assembly that can be removed andinterchanged with various different components of differentcapabilities. It is contemplated that the modular and removable assemblyportion of the headphone is designed to be a removable “puck” which maybe round, oval, hemispherical, or any other suitable shape which can bemagnetically coupled, screwed, snap fit, swage fit or otherwiseremovably connected in any other reliable mechanical and electricallyconductive manner so as to provide interchangeable functionality for theuser. These pucks can also operate independently from each other and/orfrom the headphone itself to provide independent functionality, such asby way of non-limiting examples as a conference module, image capturesystem, set-top box, image display or projector.

Such a configuration provides for the removal and connection of pucks ofdifferent capability to suit a particular user's particular applicationat a particular time, as well as providing the ability for puck modulesto be modifiable and adaptable so as to incorporate revisions, changesand additions to capability over time without requiring the user toreplace the headphone assembly. Thus a user purchasing the headphoneassembly can adapt their initial investment to a low level offunctionality and gradually increase to higher levels of functionalityfor an additional investment, or incorporate improved functionality overtime as new pucks and new features become available, without having todiscard the basic platform due to obsolescence.

In one embodiment, the device includes a headband portion designed to beworn atop a user's head spanning from one side of the user's head to theother. In some embodiments, the headband of the device includesstandalone or interoperable processing elements such as system-on-a-chip(“SoC”) devices, microprocessors, graphics processors, connectivityinterfaces (e.g., Bluetooth, Wi-Fi, NFC, etc.), and various othercomponents that can function alone or be interoperable with the pucks toaugment the capabilities of the pucks and/or connected goggles orglasses. Components of the headband can include general purposeprocessing elements utilized in all VR/AR operations. The headbandportion can additionally comprise a bi-directional bus spanning thelength of the headband. In other embodiments the headphone assembly canbe a simple mechanical frame for receiving pucks, glasses and modularplugins and contain little or no electronic components, or can just actas a wire bus for connecting components, as discussed further herein.

In some embodiments, the headband portion of the device can be connectedto an ear piece portion on each end designed to be worn proximate, over,or adjacent to a user's ears. In some embodiments, the ear piece portioncan be mechanically and communicatively or removably coupled to theheadband portion. In some embodiments, the headband can provide audiosignals to the ear piece portion and the ear piece portion. The earpiece portion can additionally include a speaker for playing audio whilea user experiences a VR or AR environment. Alternatively, the speaker(s)can be part of the removable puck.

The ear piece portion can additionally include an external mechanicaland physical connector designed to receive a puck. In some embodiments,the puck includes additional processing elements to perform variousVR/AR processing operations. In some embodiments, the ear piece portionincludes a USB connection to allow for data transfer between the puckand the ear piece and further to other components via connections in theheadband.

In one embodiment, the puck can include multiple photosensors to detectlight from a projection device and enable positional tracking. In someembodiments, the puck can detect a user's position within a physicalspace and transmit the user's position to a scene renderer which canupdate the display of a three-dimensional scene via the glasses orprojectors on the pucks, based on the user's position.

Alternatively, or in conjunction with the foregoing, the puck caninclude a gyroscope, accelerometer, and magnetometer to track the headmovements of the VR or AR device. In some embodiments, the puck candetect a user's head position and transmit the user's head position to ascene renderer which can update the display of a three-dimensional scenevia the glasses or projectors on the pucks, based on the user'sposition.

Alternatively, or in conjunction with the foregoing, the puck caninclude one or more camera devices configured to record light fields orreflectance fields of a physical space. In this embodiment, the puck canrecord one or more light fields or reflectance fields of a physicalspace and transmit the recorded images to a scene processor within or incommunication with the puck. In some embodiments, the scene processorcan be configured to stitch the recorded images together and generate areal-time three-dimensional scene from the recorded images. In someembodiments, the camera(s) can transmit images to a storage devicelocated in the puck (or headband) as a two-dimensional array oftwo-dimensional images, forming a four-dimensional light field dataset.

Alternatively, or in conjunction with the foregoing, the puck caninclude one or more microphones for recording voice commands or otheraudible signals. In one embodiment, the microphones can be configured torecord ambient noise within a physical space and adjust the volume ofaudio associated with the three-dimensional scene accordingly.

The puck is configured to be self-supporting and independently operablewhen disconnected from the headphone assembly. In some embodiments, thepuck can be configured to act as a capture device to enable holographicvideoconferencing. Specifically, the puck can be equipped with one ormore capture devices (e.g., a camera array) configured to record imagesof a physical space. In some embodiments, the puck can be equipped withone or more light sources configured to illuminate a physical space (andobjects within it) at multiple angles.

In some embodiments, the headband portion or pucks can be configuredwith additional processing elements. For example, a puck can be equippedwith an EEG monitor (or alternative brain-computer interfaces) or otherrecording apparatus. Alternatively, or in conjunction with theforegoing, a puck or headband can be equipped with one or more hapticelements.

Alternatively, or in conjunction with the foregoing, the puck caninclude one or more batteries. In some embodiments, the puck can includean external charging port for recharging the one or more batteries. Insome embodiments, the charging port can comprise a USB port while inother embodiments the charging port can comprise an inductive orcapacitive charging surface. In some embodiments, the puck can becommunicatively coupled to a flying drone device. In this embodiment,the drone device can be configured to monitor the power level of thepuck and, upon detecting a low power condition, can initiate one or morerotors and disconnect from the puck. Within a physical space, one ormore power sources can be connected to power outlets or other powersources. These power sources can include a USB output port and/or aninductive capacitive or other cordless electromagnetically coupledcharging pad or device (for example a laser powered photovoltaic cell).In this embodiment, the drone device can, upon disconnection from thepuck, detect the location of the external power source and navigate tothe external power source. Upon reaching the external power source, thedrone connects (e.g., via USB or wirelessly charging pad) to theexternal power source and charges a battery located within the drone.Upon fully charging, the drone can return to the puck, connect to thecharging port of the puck, and discharge the power to the puck, thusrecharging the puck. In alternative embodiments, the drone device can beconfigured to replace pucks with low battery levels with replacementpucks that are fully charged via a power source.

Alternatively, or in conjunction with the foregoing, the VR or AR devicecan be used in conjunction with a plurality of drone recording devices.In this embodiment, the drones can record images of a physical space atmultiple angles and transmit the images to the puck(s) for furtherprocessing. In some embodiments, the pucks can be equipped with a scenerenderer which translates the recorded images into a real-timethree-dimensional scene.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other objects, features, and advantages of thedisclosure will be apparent from the following description ofembodiments as illustrated in the accompanying drawings, in which likereference characters refer to the same parts throughout the variousviews. The drawings are not necessarily to scale, emphasis instead beingplaced upon illustrating principles of the disclosure.

FIGS. 1A through 1D depict a modular headphone assembly-structured VR/ARdevice according to some embodiments of the disclosure.

FIGS. 2A-2I depict a modular headphone-based VR/AR device assembly witha detached puck according to some embodiments of the disclosure.

FIGS. 3A-3U depict a puck according to some embodiments of thedisclosure.

FIGS. 4A-4H depict a modular headphone-based VR/AR device assembly witha detachable or movable goggle portion according to some embodiments ofthe disclosure.

FIG. 5 is a block diagram illustrating a puck according to someembodiments of the disclosure.

FIGS. 6A through 6C are block diagrams illustrating a puck according tosome embodiments of the disclosure.

FIG. 7 is a block diagram illustrating a drone-based recharging systemaccording to some embodiments of the disclosure.

FIGS. 8A and 8B illustrate drone delivery devices according to someembodiments of the disclosure.

FIG. 9 is a block diagram of a drone-based capture system according tosome embodiments of the disclosure.

FIGS. 10A-B illustrates a puck with a display configured to act as aholographic, light field display.

FIGS. 11A-C illustrate a puck with a display configured to display abody- or object-based virtual display, according to some embodiments ofthe disclosure.

FIGS. 12A-H illustrate a puck with a wand-mounted display configured toact as a holographic, light field display, according to some embodimentsof the disclosure.

FIGS. 13A-F illustrate a puck with a display configured to act as avapor-based holographic, light field display, according to someembodiments of the disclosure.

FIGS. 14A-E illustrate a vapor-based projection puck according to someembodiments of the disclosure.

DETAILED DESCRIPTION

The present disclosure will now be described further hereinafter withreference to the accompanying drawings, which form a part hereof, andwhich show, by way of illustration, certain example embodiments whichcan be implemented in whole or in part or in various combinations amongembodiments. Subject matter may, however, be embodied in a variety ofdifferent forms and, therefore, covered or claimed subject matter isintended to be construed as not being limited to any example embodimentsset forth herein; example embodiments are provided merely to beillustrative. Likewise, a reasonably broad scope for claimed or coveredsubject matter is intended. Among other things, for example, subjectmatter may be embodied as methods, devices, components, or systems.Accordingly, embodiments may, for example, take the form of hardware,software, firmware or any combination thereof (other than software perse). The following detailed description is, therefore, not intended tobe taken in a limiting sense.

Throughout the specification and claims, terms may have nuanced meaningssuggested or implied in context beyond an explicitly stated meaning.Likewise, the phrase “in one embodiment” as used herein does notnecessarily refer to the same embodiment and the phrase “in anotherembodiment” as used herein does not necessarily refer to a differentembodiment. It is intended, for example, that claimed subject matterinclude combinations of example embodiments in whole or in part.

In general, terminology may be understood at least in part from usage incontext. For example, terms, such as “and”, “or”, or “and/or,” as usedherein may include a variety of meanings that may depend at least inpart upon the context in which such terms are used. Typically, “or” ifused to associate a list, such as A, B or C, is intended to mean A, B,and C, here used in the inclusive sense, as well as A, B or C, here usedin the exclusive sense. In addition, the term “one or more” as usedherein, depending at least in part upon context, may be used to describeany feature, structure, or characteristic in a singular sense or may beused to describe combinations of features, structures or characteristicsin a plural sense. Similarly, terms, such as “a,” “an,” or “the,” again,may be understood to convey a singular usage or to convey a pluralusage, depending at least in part upon context. In addition, the term“based on” may be understood as not necessarily intended to convey anexclusive set of factors and may, instead, allow for existence ofadditional factors not necessarily expressly described, again, dependingat least in part on context.

The present disclosure is described below with reference to blockdiagrams and operational illustrations of methods and devices. It isunderstood that each block of the block diagrams or operationalillustrations, and combinations of blocks in the block diagrams oroperational illustrations, can be implemented by means of analog ordigital hardware and computer program instructions. These computerprogram instructions can be provided to a processor of a general purposecomputer to alter its function as detailed herein, a special purposecomputer, ASIC, or other programmable data processing apparatus, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, implement thefunctions/acts specified in the block diagrams or operational block orblocks. In some alternate implementations, the functions/acts noted inthe blocks can occur out of the order noted in the operationalillustrations. For example, two blocks shown in succession can in factbe executed substantially concurrently or the blocks can sometimes beexecuted in the reverse order, depending upon the functionality/actsinvolved.

These computer program instructions can be provided to a processor of: ageneral purpose computer to alter its function to a special purpose; aspecial purpose computer; ASIC; or other programmable digital dataprocessing apparatus, such that the instructions, which execute via theprocessor of the computer or other programmable data processingapparatus, implement the functions/acts specified in the block diagramsor operational block or blocks, thereby transforming their functionalityin accordance with embodiments herein.

For the purposes of this disclosure a computer readable medium (orcomputer-readable storage medium/media) stores computer data, which datacan include computer program code (or computer-executable instructions)that is executable by a computer, in machine readable form. By way ofexample, and not limitation, a computer readable medium may comprisecomputer readable storage media, for tangible or fixed storage of data,or communication media for transient interpretation of code-containingsignals. Computer readable storage media, as used herein, refers tophysical or tangible storage (as opposed to signals) and includeswithout limitation volatile and non-volatile, removable andnon-removable media implemented in any method or technology for thetangible storage of information such as computer-readable instructions,data structures, program modules or other data. Computer readablestorage media includes, but is not limited to, RAM, ROM, EPROM, EEPROM,flash memory or other solid state memory technology, CD-ROM, DVD, orother optical storage, magnetic cassettes, magnetic tape, magnetic diskstorage or other magnetic storage devices, or any other physical ormaterial medium which can be used to tangibly store the desiredinformation or data or instructions and which can be accessed by acomputer or processor.

For the purposes of this disclosure a “network” should be understood torefer to a network that may couple devices so that communications may beexchanged, such as between a server and a client device or other typesof devices, including between wireless devices coupled via a wirelessnetwork, for example. A network may also include mass storage, such asnetwork attached storage (NAS), a storage area network (SAN), or otherforms of computer or machine readable media, for example. A network mayinclude the Internet, one or more local area networks (LANs), one ormore wide area networks (WANs), wire-line type connections, wirelesstype connections, cellular or any combination thereof. Likewise,sub-networks, which may employ differing architectures or may becompliant or compatible with differing protocols, may interoperatewithin a larger network. Various types of devices may, for example, bemade available to provide an interoperable capability for differingarchitectures or protocols. As one illustrative example, a router mayprovide a link between otherwise separate and independent LANs.

A communication link or channel may include, for example, analogtelephone lines, such as a twisted wire pair, a coaxial cable, full orfractional digital lines including T1, T2, T3, or T4 type lines,Integrated Services Digital Networks (ISDNs), Digital Subscriber Lines(DSLs), wireless links including satellite links, or other communicationlinks or channels, such as may be known to those skilled in the art.Furthermore, a computing device or other related electronic devices maybe remotely coupled to a network, such as via a wired or wireless lineor link, for example.

For purposes of this disclosure, a “wireless network” should beunderstood to couple client devices with a network. A wireless networkmay employ stand-alone ad-hoc networks, mesh networks, Wireless LAN(WLAN) networks, cellular networks, or the like. A wireless network mayfurther include a system of terminals, gateways, routers, or the likecoupled by wireless radio links, or the like, which may move freely,randomly or organize themselves arbitrarily, such that network topologymay change, at times even rapidly.

A wireless network may further employ a plurality of network accesstechnologies, including Wi-Fi, Long Term Evolution (LTE), WLAN, WirelessRouter (WR) mesh, or 2nd, 3rd, or 4th generation (2G, 3G, 4G or 5G)cellular technology, or the like. Network access technologies may enablewide area coverage for devices, such as client devices with varyingdegrees of mobility, for example.

For example, a network may enable RF or wireless type communication viaone or more network access technologies, such as Global System forMobile communication (GSM), Universal Mobile Telecommunications System(UMTS), General Packet Radio Services (GPRS), Enhanced Data GSMEnvironment (EDGE), 3GPP Long Term Evolution (LTE), LTE Advanced,Wideband Code Division Multiple Access (WCDMA), Bluetooth, Bluetooth LowEnergy (BLE), 802.11b/g/n, near-field wireless, or the like. A wirelessnetwork may include virtually any type of wireless communicationmechanism by which signals may be communicated between devices, such asa client device or a computing device, between or within a network, orthe like.

A computing device may be capable of sending or receiving signals, suchas via a wired or wireless network, or may be capable of processing orstoring signals, such as in memory as physical memory states, and may,therefore, operate as a server. Thus, devices capable of operating as aserver may include, as examples, dedicated rack-mounted servers, desktopcomputers, laptop computers, set top boxes, integrated devices combiningvarious features, such as two or more features of the foregoing devices,or the like. Servers may vary widely in configuration or capabilities,but generally a server may include one or more central processing unitsand memory. A server may also include one or more mass storage devices,one or more power supplies, one or more wired or wireless networkinterfaces, one or more input/output interfaces, or one or moreoperating systems, such as Windows Server, Mac OS X, Unix, Linux,FreeBSD, or the like.

FIGS. 1A through 1D depict a modular headphone assembly-structured VR/ARdevice according to some embodiments of the disclosure.

As illustrated in FIGS. 1A through 1D, a VR/AR device 100 includes aheadband 102. In some embodiments, headband 102 comprises a flexible orelastically deformable hemispherical strap-like element designed to beworn atop a user's head. In some embodiments, the headband 102 cancomprise multiple, adjustable portions such that a user can expand orcontract the headband 102 to fit snuggly against the user's head, andcan be cushioned in various locations along the head-engaging span forcomfort. Each end of headband 102 can be connected to an attachmentmember 116 which, in turn, is connected to an ear piece portion 120. Insome embodiments, an air gap is present between puck 104 and ear pieceportion 120. In this embodiment, the air gap can provide cooling airflow and heat dissipation from puck 104. In this embodiment, puck 104can include one or more ventilation slots or heat sink fins orstructures facing the air gap to allow for passive heat transfer.

In some embodiments, headband 102 can be connected to attachment member116 via rotatable disc 124 which can be spring or tension loaded toallow relative retaining motion for fit and comfort proximate, around,or over the wearer's ear. In this embodiment, rotatable disc enables theinward and outward movement of ear piece portion 120. In someembodiments, arms 106 a, b, crossbar 108, and goggle portion 110 can beremoved from the device 100. Thus, rotatable disc 124 allows formovement of ear piece portion 120 (and puck 104) akin to the movement oftraditional headphones. In one embodiment, headband 102 can include allnecessary components to render VR or AR scenes to goggle portion 110. Inthis embodiment, puck 104 can be removed (either permanently ortemporarily). In some embodiments, headband 102 can include a USB orsimilar connector to allow for the connection of additional processingdevices. For example, a user can connect a device containing one or moreprocessing elements (e.g., additional VR/AR processing elementsdescribed herein) having a male USB-C® connection into a female USB-C®connection port present on the top of headband 102 as more fullyillustrated in FIG. 2H.

In some embodiments, ear piece portion 120 can include a speaker 128directed inward toward a user's ear. Ear piece portion 120 can in turnbe connected to a foam ear cushion 118 designed to cushion the assemblyagainst the wearer's head. The outer surface of ear piece portion 120 isillustrated more fully in FIGS. 2A-I and the accompanying description.In some embodiments, ear piece portion 120 may not include additionalprocessing elements and can be used primarily to route communicationsbetween the active devices forming other parts of the assembly, such aspucks or components removably mounted to the headband 102. Inalternative embodiments, additional processing elements can be placedwithin ear piece portion 120. For example, ear piece portion 120 can beequipped with additional haptic or audio devices to enhance a VR or ARexperience (e.g., an electromechanical vibrator or bone-conductionsub-woofer).

In one embodiment, headband 102 can be equipped with one or morepermanent or replaceable processing elements. In one embodiment,headband 102 can include one or more SoC devices, microprocessors,graphic processing units (GPU), and other processing devices such asnetworking devices, EEG sensors, brain-computer interface (“BCI”)devices, haptic devices, etc. Each of the processing elements within theheadband 102 can be connected to a bus that spans the length of theheadband. In some embodiments, the bus can be routed to the ear pieceportion 120 and to arm 106 and crossbar 108. Thus, as illustrated, abi-directional communications bus forms a circular or semi-circular busconnecting each electronic component in device 100. The bus allows forcommunication between processing devices in the headband 102, puck 104,crossbar 108, and goggle portion 110 as discussed in more detail herein.

Attached to the ear piece portion 120 is a detachable processing device,or puck, 104. As described in more detail herein, puck 104 comprises afull-fledged mobile computing device that can communicate with otherelements (e.g., headband 102, crossbar 108, and goggle portion 110) viaa bi-directional bus. Additionally, or alternatively, puck 104 can bebattery-powered and can be configured to operate independent of othercomponents of device 100.

In one embodiment, puck 104 can be physical attached to and detachedfrom the ear piece portion 120. In one embodiment, puck 104 can beconnected to ear piece portion 120 via a magnetic coupling, screwcoupling, snap fit coupling, swage fit coupling, or any other reliablemechanical and electrically conductive manner so as to provide removableand interchangeable functionality for the user. Examples of physicalconnection mechanisms are discussed more fully in connection with FIGS.2A-I.

In one embodiment, in addition to being physically connected to earpiece portion 120, puck 104 can additionally be communicatively coupledto ear piece portion 120 via one or more USB-C® connectors placed on theinward facing side of puck 104. In one embodiment, puck 104 and earpiece portion 120 can each include a female USB-C® receptacle and can becommunicatively coupled with a male-to-male USB-C® connector asillustrated in FIGS. 2F-2G. In alternative embodiments, puck 104 can bedesigned such that a male USB-C® connector present on the inside of puck104 can be inserted into a female USB-C® receptacle present on the earpiece portion 120. In alternative embodiments, puck 104 can beconfigured with a female USB-C® receptacle and ear piece portion 120 canbe configured with a male USB-C® connector. The use of a USB-C®connection allows for communications between the puck 104 and theheadband 102 and goggle portion 110, using the bus discussed previously.In alternative embodiments, other communications protocols can be usedto connect puck 104 with ear piece portion 120. For example, near fieldcommunications, BLE or other wireless coupling methodologies can be usedalone or in combination.

Located on the outside of puck 104 are multiple openings, apertures ormounting positions referred to herein interchangeably as “rivets” 112a-112 i. In various embodiments, rivets 112 a-112 i can house or retainvarious components for performing VR or AR-specific operations. Forexample, rivets 112 a-112 i can house capture devices or trackingdevices or combinations thereof. In some embodiments, puck 104 can beequipped with only capture devices or with only tracking devices in orat each rivet 112 a-112 i. In alternative embodiments, puck 104 can beconfigured with both capture devices and tracking devices at or in eachof rivets 112 a-112 i.

In one embodiment, a capture device can comprise a camera and/or an LED.In this embodiment, the LED can be utilized to illuminate a physicalspace while the camera records images at one or more angles. In someembodiments, an LED and camera can be combined into a single rivet,while in alternative embodiments (discussed herein) LEDs and cameras canbe placed in individual rivets at various locations on the puck. In someembodiments, other light sources other than LEDs can be used in place ofLEDs. In some embodiments, a light source placed in or at rivets 112a-112 i can comprise a polarized light source, unpolarized light source,laser diode, infrared (IR) source or combinations thereof. As usedherein a light source can be a device that emits electromagnetic orphotonic energy in visible or invisible wavelengths. In one embodiment,images captured by capture devices can be used to collect image datathat can be used for generating content, including VR content. In someembodiments, the images captured by the multiple capture devices in orat rivets 112 a-112 i can be stored in memory present within the puck104 and used for later display as a three-dimensional scene via crossbar108 and goggle portion 110. In some embodiments, the cameras can befitted with wide angle lenses or fisheye lenses. Thus, in someembodiments, puck 104 can be configured as a portable light field orreflectance field capture device and can transmit light field orreflectance field image data to goggle portion 110 or to other deviceson or in communication with the device 100. In one embodiment, puck 104can allow a user to view a three-dimensional rendering of a space inreal-time or near-real time. To enable this operation, puck 104 can beconfigured with one or more processors to process light field orreflectance field images or to send some or all raw light field data toan external device and receive a stream of further processed datarepresenting the VR or AR scene to be rendered. Additional details ofthe operation of capture devices in connection with light field andreflectance field capture is described more fully in connection withFIGS. 6A through 6C, although persons of skill will recognize thathaving light field or reflectance field capture devices mounted at eachside of a user's head offers great advantages in the creation, captureand rendering of truly accurate and immersive VR/AR experiences.

In some embodiments, the cameras are configured to capture thereflectance fields of a subject and/or space. In this embodiment, thecameras capture images of the subject/space from multiple viewpointsusing a dense sampling of incident illumination provided by lightsources on the puck. The cameras transmit these images to a processorwhich constructs a reflectance function image for each observed imagepixel from its values over the space of illumination directions. Aprocessor is then able to generate images of the space/subject from theoriginal viewpoints in any form of sample or computed illumination. Inorder to change viewpoints, the processor can then utilize a constructedmodel of the subject/space to estimate the appearance of the reflectancefunctions for different and/or new viewpoints. In some embodiments, twoor more pucks can be used to capture reflectance fields. In thisembodiment, pucks can communicate between each other to identify theprecise location of light sources present on the puck cameras. In someembodiments, two or more pucks can be placed at varying distances or canbe carried by one or more drone devices as discussed in connection withFIGS. 8A, 8B, and 9. In some embodiments, multiple pucks are capable ofsynchronizing cameras and/or light sources during operation.

Puck 104 can additionally include multiple tracking devices in rivets112 a-112 i. In one embodiment, a tracking device can comprise aphotosensor device configured to detect light emitted from a positionaltracking base station. In alternative embodiments, tracking devices cancomprise a depth camera. In these embodiments, puck 104 can beconfigured to track the position of a user while the user views a VR orAR display. Alternatively, or in conjunction with the foregoing, puck104 can additionally include processing elements to track the headmovements of a user. Details on the operation of tracking devices aredescribed more fully in connection with FIGS. 6A through 6C. Variouslayouts of VR/AR processing devices in rivets 112 a-112 i are describedmore fully in connection with FIGS. 3A-R, and in particular FIG. 3H.

Puck 104 can additionally include one or more projection devices 130 a,130 b. In one embodiment, projection devices 130 a, 130 b can compriseswivelable or positionable pico projector modules, pico projectors beingsmall devices that can project images in the area surrounding the picoprojector, generally laser based. In some embodiments, projectiondevices 130 a, 130 b can include or intercommunicate with optional depthsensors. In some embodiments, projection devices 130 a, 130 b caninclude an RGBZ module. Projection devices 130 a, 130 b can beconfigured to project light field scenes or other visible video or imageinformation into a physical space, generally onto one or more surfacesin the space.

In one embodiment, projection devices 130 a, 130 b can be utilized whilea user is wearing the device 100. In alternative embodiments, projectiondevices 130 a, 130 b can be used when the puck 104 is detached from thedevice 100. In alternative embodiments, projection devices 130 a, 130 bon one puck being used while detached while the user utilizes the device100. In some embodiments, puck 104 can include at least three projectorsto provide a wide degree projection that from a user's perspective seemsto encompass most or all of their field of view. In this embodiment,projectors can be spaced according to the range of projection providedby each projector. In some embodiments, projection devices 130 a, 130 bcan have an electronically or mechanically controlled skew mechanismallowing for projection devices 130 a, 130 b to be reoriented as needed.In some embodiments, projection devices 130 a, 130 b can be utilized inaugmented reality applications. For example, projection devices 130 a,130 b can be used with a mapping application wherein a route isprojected downward (e.g., onto a sidewalk) while a user is in motion(for example to show projected footprints or line(s) to follow).

In embodiments, puck 104 additionally includes a display device 114. Inone embodiment, display device 114 can comprise an OLED touchscreendisplay. In some embodiments, display device 114 can be communicativelycoupled to one or more processors such as video processors or graphicsprocessing units (GPUs) embedded within puck 104. In one embodiment,display device 114 can display information regarding the processingoperations of puck 104. In one embodiment, display device 114 candisplay the view a user sees in display device 114. In alternativeembodiments, display device 114 can display a QR code that identifiesthe user. For example to promote social networking the display 114 candisplay a mood or status or QR code reflecting same, so as to alertother in proximity of the user's social interactivity, desires oravailability. In alternative embodiments, a QR code can be printed onpuck 104 itself. In alternative embodiments, display device 114 candisplay a song or video being listened to or viewed, respectively, bythe user. In alternative embodiments, display device 114 can displaynotifications or information regarding the user. For example, displaydevice 114 can display that a user is capable of hearing another userdespite wearing the puck over the user's ear. As discussed previously,puck 104 can be configured to display information on display device 114even while disconnected from device 100. In these embodiments, puck 104can operate as a handheld, networked computing device or set-top box tocontrol content or generate content to be displayed on a remote displayalone or in concert with attached goggles. In some embodiments, puck 104can additionally include a touch-sensitive scroll wheel 126. In thisembodiment, scroll wheel 126 can comprise a capacitive sensor, or aring-shaped OLED touchscreen display circling display 114, or othertouch sensitive sensing mechanism to detect a user's finger or handcontact for device control.

In embodiments, puck 104 additionally includes an input device 122. Inone embodiment, input device 122 can comprise a trackball deviceconfigured to control interaction with display device 114. In someembodiments, input device 122 can include haptic rumble, pressuresensitivity, and/or modal click functionalities. In some embodiments,input device 122 can additionally include one or more navigationalbuttons (e.g., a “forward” or “back” button) to enable a user tonavigate through user interfaces displayed on display device 114. Inalternative embodiments, puck 104 can be equipped with an expandable“accordion” keyboard that expands outwardly from the center of puck 104.Examples of an accordion keyboard are described more fully in connectionwith FIG. 3O. Alternatively, or in conjunction with the foregoing, puck104 can be configured to receive an external mouse or keyboard input viaa USB input port. Puck 104 can additionally include one or more pushbuttons 132.

As discussed previously, puck 104 may be configured to transmit data togoggle portion 110 via arm 106 and crossbar 108. In one embodiment, arm106 may comprise a bus connecting ear piece portion 120 (and thus, puck104) to crossbar 108. In alternative embodiments, arm 106 canadditionally be configured with additional processing devices (e.g.,devices to support head tracking, position tracking, or light fieldcapture). In some embodiments, a puck 104 on the side of device 100 canbe configured to drive a single OLED display 114. For example, a puck onthe left side of device 100 can be configured to drive an OLED displayon the left side of device 100. Likewise, a puck on the right side ofdevice 100 can drive an OLED display on the right side of device 100, ora single puck can control both sides.

As illustrated, goggle portion 110 is connected to ear piece via arm106. In one embodiment, goggle portion 110 can be detachable asillustrated in FIG. 4B. Goggle portion 110 can be connected to arm 106via crossbar 108. In one embodiment, crossbar 108 can include one ormore processors or other components for controlling the goggle portion110. For example, crossbar 108 can include and HDMI to MIPI converter, aUSB-C® controller, a dedicated microcontroller, cache memory, eyetracking processor, and other components. However, crossbar 108 is notrequired to include processing elements for generating three-dimensionalscenes. Instead, three-dimensional scenes can be processed andtransmitted to goggle portion 110 from one or more pucks 104 or headbandbased processors or external sources using scene data collected bycomponents of device 100. Thus, in some embodiments, the weight ofcrossbar 108 can be reduced and processing functionality can beoffloaded to one or more pucks 104 and/or headband 102.

In some embodiments, puck 104 can be additionally configured with one ormore external hooks located on the exterior perimeter of the puck 104.In this embodiment, the hooks allow for automated detachment of the puck104 from the ear piece portion 120. In alternative embodiments, puck 104can include a slot separating the puck 104 from ear piece portion 120.In this embodiment, a drone device can be configured to remove the puck104 from ear piece portion 120 by inserting a fork-lifting or hookmember into the slot and moving away from the ear piece portion 120,thus detaching the puck 104 from the ear piece portion 120. Inalternative embodiments, puck 104 can be configured with an outwardfacing charging port (e.g., a USB port or inductive charging pad) tofacilitate re-charging the puck via a drone re-charging mechanismdiscussed more fully herein.

As illustrated above, the device 100 comprises a highly modular VR/ARdevice. In operation, various components of the device 100 can beremoved, replaced, or upgraded as needed. Thus, users are not requiredto replace the entire device upon failure of a single component or uponupgrades to technology. For example, in one embodiment headband 102 cancomprise core processing components to display VR or AR scenes viagoggle portion 110. In this embodiment, a puck 104 can be utilized toadd additional VR or AR functionality (e.g., position tracking, lightfield capture, etc.) to a base processing functionality. Additionally,in alternative embodiments, pucks 104 can be replaced or interchangedwith other pucks having differing components as required by the userunder differing circumstances.

FIGS. 2A-2I depict a modular headphone-based VR/AR device with adetached puck according to some embodiments of the disclosure.

As illustrated in FIGS. 2A-F, puck 104 can be removed or detached fromdevice 100 upon physical manipulation of the user or via an automatedremoval procedure (e.g., drone removal or switch activation or voicecommand). FIGS. 2A-F illustrate the covered portion of ear piece portion120. In this embodiment, puck 104 can be communicatively coupled todevice 100 via an inductive connection between puck 104 and ear pieceportion 120 or via a wireless connection between puck 104 and ear pieceportion 120. In alternative embodiments (discussed below), a wiredconnection can be used to connect puck 104 to device 100.

As illustrated in FIG. 2B, docking surface 202 of ear piece portion 120can include a female USB-C® receptacle 204. Likewise, puck 104 caninclude a female USB-C® receptacle 206. Additionally, puck 104 caninclude multiple support arms 208 a-c configured to retain the positionof the puck 104 upon attachment to docking surface 202 and or act asmounting surfaces for components or sensors. Alternatively, or inconjunction with the foregoing, arms 208 a-c can be used to align thepuck 104 during automated docking and undocking. Notably, arms 208 a-cmay be optional components as illustrated in FIGS. 3P through 3R.

FIGS. 2C and 2D illustrate alternate views of the VR/AR device 100discussed in connection with FIGS. 2A and 2B. FIG. 2E illustrates thedetachment of puck 104 when a goggle portion is detached from the VR/ARdevice 100.

FIG. 2F illustrates the use of a male-to-male USB-C® connector 210 tocommunicatively (and, in some embodiments, physically) connect puck 104to ear piece portion 120 via docking surface 202. As illustrated in moredetail in FIG. 2G, connector 210 includes two opposing male USB-C®connectors 212, 214 separated by a flange 216. In this embodiment, afirst male end 212 of connector 210 can be inserted into puck 104 whilethe opposing end 214 can be inserted into docking surface 202. Asdiscussed, the use of connector 210 can allow for the physical andcommunicative coupling of puck 104 to a VR/AR device 100.

As illustrated in FIG. 2H, a VR/AR device 100 can include a modularheadband extension 218. Extension 218 can comprise a full-fledgedprocessing device as described in connection with FIG. 5. In someembodiments, extension 218 can further include VR/AR-specific processingelements as discussed in connection with FIGS. 6A through 6C. Asillustrated, module headband extension 218 can connect to headband 102via a USB connection 220 a, 220 b. In alternative embodiments, extension218 can be connected to headband 102 via a wireless connection such as aBluetooth, NFC, or Wi-Fi connection.

FIG. 2I illustrates a puck 104 and a removable processor component 222according to some embodiments of the disclosure. In some embodiments, aremovable processor component 222 can be placed in between puck 104 anda mounting plate. As described more fully in connection with FIGS. 6Band 6C, a processor component 222 can comprise a component fitted tonest between a mounting plate and the rear side of puck 104. In thisembodiment, the processor component 222 can include the main processingcomponents used by a VR/AR device and can connect to the headset via USBports 206 and 224. In this embodiment, puck 104 can include fewerelectronics and can offload processing to the processor component 222via USB ports 206 and 224. For example, puck 104 may only include anFPGA to drive VR/AR components and may forward all data to processorcomponent 222 via USB ports 206, 224 for further processing. Thisconfiguration facilitates the swapping out of major processingcomponents in a modular manner thus further protecting a user'sinvestment in the device of the present disclosure, since the pucks canbe modified with improved processing power without replacing the entirepuck.

FIGS. 3A-3R depict a puck according to some embodiments of thedisclosure.

FIGS. 3A and 3B illustrate a top view of a puck and FIG. 3C illustratesa perspective view of a puck. In the illustrated embodiments, a puck 104includes a plurality of “rivets” 112 a-112 i. As discussed previously,rivets 112 a-112 i can include VR or AR specific processing elementssuch as cameras, light sources, photosensors, projectors, microphones orpick-ups for collecting ambient sound or user voice commands, and/orother I/O devices or sensors now known or hereafter to become known. Anexample arrangement of VR/AR processing elements in rivets 112 a-112 iis described more fully in connection with FIG. 3H.

FIGS. 3D-3G and 3I-3J illustrate a puck with a display removed. Asillustrated in FIG. 3C, display 114 can be configured to be removed frompuck 104 by a user. In one embodiment, display 114 can be connected to ahinge allowing display 114 to “flip” upward, away from puck 104. Inalternative embodiments, display 114 can be connected to puck 104 via aball and socket joint, that may be motor controlled. In alternativeembodiments, display 114 can be configured to slide upward (away frompuck 104) and outward (away from the center of puck 104). In someembodiments, display 114 can configured to be completely disconnectedfrom puck 104. In this embodiment, display 114 can include a wirelessnetwork interface (e.g., a Bluetooth interface) to allow forcommunication between puck 104 and display 114 when disconnected.

In some embodiments, display 114 can comprise an OLED touchscreendisplay. In some embodiments, display 114 can be double-sided. That is,the display 114 can have two display surfaces on each side.

In one embodiment the OLED display 114 can be mounted on a mechanicalmotor driven assembly such that when the OLED display 114 is moved froma position flush with or parallel to the surface of the puck 104 to aposition orthogonal or semi-orthogonal to the surface of the puck 104,the OLED display 114 can be driven to move around an axis orthogonal orangled to the surface of the puck 104. This permits the OLED display 114to be moved in a manner similar to a radar dish. This allows the OLEDdisplay 114 to track the movement of the user so that the display faceof the OLED display 114 will always be in a proper orientation withrespect to the eyes of the observer. Thus, the OLED display 114 moves tofollow the user in the proper orientation. As discussed herein, theheadphone assembly contains, in various embodiments, position trackingcomponents, whether in an inside-out, or outside-in configuration. Thisposition data can be used to drive the OLED.

For example if the puck 104 is held in a user's hand or placed on asurface proximate the user, the projectors on the puck 104 and/or thecameras on the puck 104 or the other position sensing components of theheadphone assembly can be used alone or in concert to provide the finepositioning of the user relative to the OLED display. As the headphoneassembly or connected or free-standing puck(s) determines the properorientation of the user's face relative to the OLED display 114, thatinformation can be fed wirelessly to a microcontroller in the puck 104that is connected to a small micro-motor or step-motor that can controlthe OLED display 114 in response to the changes in orientation of theuser relative to the surface position of the OLED display 114. In thisway, the OLED display 114 can constantly track the user's movement andbe rotated around an axis orthogonal or at an angle to the puck 104 tofollow the user as the user moves relative to the puck 104, whether in ahandheld position or whether the puck 104 is positioned on a surfaceproximate the user.

Upon movement or removal of display 114, cavity 302 can be exposed. Inone embodiment, cavity 302 can include a keyboard or other input device.In some embodiments, cavity 302 can include an additional touchscreendisplay (e.g., an OLED touchscreen display). In some embodiments, cavity302 can include a touch ball or trackpad. Additionally, as discussedpreviously, in embodiments puck 104 includes a circular or other shapetouchscreen controller 126. Thus, in some embodiments, upon removal ofdisplay 114, puck 104 can include a touchscreen controller 126 to detectuser finger movements to perform “scrubbing” operations and a trackpador touch ball to perform other movement operations.

FIGS. 3H and 3K illustrate a puck including a retractable hookmechanism. FIGS. 3H and 3K additionally illustrate specificconfigurations of VR/AR devices.

As illustrated in FIG. 3H, a puck 104 can be equipped with a drone hook312. In one embodiment, drone hook 312 can be configured to slide intoand out of puck 104 in response to a force exerted inward on drone hook312 to the center of puck 104 and outward on drone hook 312 from thecenter of puck 104, respectively. Drone hook 312 can be utilized todetach the puck 104 from a VR/AR device 100.

Additionally, as discussed previously, puck 104 includes a plurality ofrivets or openings 304 a-c, 306 a-c, and 308 a-d. In one embodiment,rivets 304 a-c can be configured to house a polarized light source,rivets 306 a-c can be configured to house an unpolarized light source,and rivets 308 a-d can be configured to house cross-polarized stereocamera pair modules and, optionally, depth sensors. In some embodiments,a cross-polarized stereo camera pair module can comprise multiplediscrete cameras including a wide-angle fish-eye camera and adjacentcameras. In some embodiments, projection devices 130 a-c can optionallyinclude cross-polarized stereo camera pair modules and optional depthsensors.

FIGS. 3G-3H additionally illustrate an alternative input mechanism. Asdiscussed previously, puck 104 can be equipped with a trackball orsimilar input device 122. In the illustrated embodiment, puck 104 canadditionally be equipped with a back button 310 a and forward (or next)button 310 b. In some embodiments, these buttons 310 a, 310 b cancomprise tactile push buttons. In some embodiments, buttons 310 a, 310 bcan additionally include haptic rumble, pressure sensitivity, and/ormodal click functionalities.

FIG. 3K illustrates puck 104 with a drone hook 312 in an extendedposition. As discussed previously, drone hook 312 can be extendedoutward from puck 104 manually or by a drone device (not illustrated).In the embodiment illustrated in FIG. 3K, drone hook 312 can comprise aflexible material to allow drone hook 312 to be manipulated by a dronedevice. In alternative embodiments, drone hook 312 can additionallycomprise a hinge or similar apparatus at its base (closest to puck 104)in order to allow forward and backward angular movement of the dronehook while the puck 104 is attached to a VR/AR device.

FIG. 3L illustrates a puck 104 configured in a “desktop” mode. In thismode, drone hook 312 is rotated toward or away from the underside ofpuck 104. By rotating drone hook 312 into various positions, puck 104can be placed on a flat surface laid flat or angled using the hook as a“kick-stand” with the puck utilized as a light field projector usingprojection devices 130 a, 130 b, and 130 c (130 c not visible asillustrated). As discussed previously, puck 104 can include at leastthree projectors spaced 120 degrees apart to provide a wide angle ofprojection. In this embodiment, projectors can be spaced according tothe range of projection provided by each projector. In some embodiments,projection devices 130 a-c can have an electronically controlled skewmechanism allowing for projection devices 130 a, 130 b, 130 c to bereoriented as needed. In some embodiments, projection devices 130 a-ccan be utilized in augmented reality applications, or for videoconferencing. For example, projection devices 130 a-c can be used with amapping application wherein a route is projected downward (e.g., onto asidewalk) while a user is in motion.

FIG. 3M illustrates the inner side of puck 104. As discussed previously,the inner side of puck 104 can include a female USB-C® connector 206.Additionally, puck 104 can include an addition projection device 130 don the inner side of puck 104. Projection device 130 d can be similar indesign to projection devices 130 a-c, previously discussed. By providinga project device 130 d on the inner side of puck 104, puck 104 may beable to project a light field display both forward and backward whenbeing carried by a drone device via drone hook 312. In some embodiments,projection device 130 d can additionally include one or more crosspolarized stereo camera pair modules to enable image capture duringflight. In alternative embodiments, inner side of puck 104 canadditionally include one or more light sources.

FIG. 3N illustrates a puck connected to a charging device. Asillustrated, puck 104 can be recharged by placing puck 104 on chargingdevice 314. In one embodiment, charging device 314 can comprise aninductive charging device (e.g., a Qi charging device) or other wirelesscoupled powering solution. In alternative embodiments, charging device314 can include a male USB-C® connector that can connect to connector206 on puck 204 to charge puck 104. Charging device 314 can be connectedto an external power source (e.g., a power outlet or computing device)via cable 316. In the illustrated embodiment, puck 104 can be fullyfunctional while charging on charging device 314. Thus, when placed oncharging device 314, puck 104 can continue to perform VR/AR operationssuch as light stage capture, projection, etc.

FIG. 3O illustrates a puck including an expandable keyboard. Asdiscussed previously, a user can lift display 114 to expose cavity 312.In the illustrated embodiment, upon exposing cavity 312 the user may beable to extend an accordion-style keyboard 318 from puck 102. In oneembodiment, keyboard 318 can be communicatively coupled to puck 104 andallow for the input of keyboard signals to manipulate display 114. Inthe illustrated embodiment, keyboard 318 can be configured to be foldedwhile display 114 covers cavity 312. Upon removing display 114, keyboard318 can automatically expand, or can expand upon activation by the user.

FIGS. 3P through 3R illustrate depict a further embodiment of a puckincluding rotatable projectors according to some embodiments of thedisclosure.

As illustrated in FIG. 3P, puck 104 includes projectors 320 a-c. In oneembodiment, projectors 320 a-c can comprise swivelable or positionablepico or micro projector modules, pico projectors being small devicesthat can project images in the area surrounding the pico projector,generally laser based. Projectors 320 a-c are connected to puck 104 viaa circular, band 322 circling puck 104. Respective conductive elementsin the band 322 mate with sliding respective contacts on the projectorsto create an electrical path between the projector and puck for powerand data transfer.

In one embodiment, band 322 includes multiple conductive traces circlingthe puck 104. In this embodiment, each projector 320 a-c can include oneor more conductive pads that align with a single conductive trace. Inthis manner, data can be transmitted from processing devices in puck 104to individual pucks. In some embodiments, band 322 can include one ormore conductive areas placed at pre-defined intervals around band 322.In this embodiment, projectors 320 a-c are rotated around band 322 untila conductive area on the projector contacts the conductive area on theband 322. Upon such meeting, data can be transmitted to and fromprojectors 320 a-c.

In some embodiments, puck 104 is configured to control the direction andprojection of projectors 320 a-c via the conductive band 322.Additionally, due to the conductive contact, projectors 320 a-c can bemoved along band 322 and, thus, can be placed at various points alongband 322. In some embodiments, projectors 320 a-c can be moved eithermanually or programmatically.

As illustrated in FIG. 3Q, a projector 602 a is connected to band 322via a C-shaped base connector 324 a. As illustrated, base connector 324a allows for the movement of a projector 320 a along the z-axisillustrated in the inset legend. Base connector 324 a is connected toconnector 326 a. In this embodiment, connector 326 a allows for therotation of projector 320 a about the x-axis illustrated in the insetlegend. Side connectors 328 a, 328 b are connected to connector 326 aand extend outwardly from puck 104. Side connectors 328 a, 328 b areconnected to projector 320 a via a rotatable joint. As illustrated, sideconnectors 328 a, 328 b allow for movement of projector 320 a along they-axis illustrated in the inset legend. In some embodiments, asmall-cell battery can be placed behind projectors 320 a-c in order toprovide supplemental or sole power to the projectors 320 a-c rather thantaking power from the puck.

FIG. 3R illustrates a configuration of projectors according to someembodiments of the disclosure. As illustrated, projectors 320 a-c can berotated to a one portion of the band 322. For example, in FIG. 3R,projectors 320 a-c are rotated towards the front of puck 104, the frontcomprising the portion facing forward and outward from the side of auser's face. In one embodiment, a VR/AR device 100 can be equipped witha simple curved reflective visor at the position of glasses or goggles108. In this embodiment, the projection of a light field on thereflective visor results in a holographic display of the light fieldbefore a user without the need for active VR/AR glasses or goggles. Inthis embodiment, projectors 320 a-c can thus be used to provide ARexperiences without the need for an OLED display or projectors in theglasses or goggles.

Specific directions are not intended to be limited, and projectors 320a-c can be pointed in any direction to provide a full light field orreflectance field projection in a physical space. In some embodiments,projectors 320 a-c can be pointed at a curved, reflective surface otherthan the goggles. In this embodiment, the projection of the light fieldusing two pucks can provide a full light field projection by overlappingprojections of projectors 320 a-c.

FIGS. 3S, 3T, and 3U illustrate alternative embodiments of a puckaccording to some embodiments of the disclosure.

In the illustrated embodiments, the puck 104 is equipped with multipleUBS-C® connectors 330A, 330B, 330C. Although three UBS-C® connectors areillustrated, the disclosed embodiments are not intended to be limited toonly three connectors and more or fewer UBS-C® connectors may be used.While described primarily in the context of UBS-C® connectors, any othertype of data connectors may be used such as, but not limited to,LIGHTNING® connectors, USB-A, UBS-C®, FIREWIRE®, or other connectortypes.

Each connector 330A, 330B, 330C comprises a housing that includes areceptacle 333A, 333B, 333C formed as part of the connector housing. Inone embodiment, a receptacle includes one or more connection points(e.g., copper or similarly conductive connection points) forfacilitating data transfer between the puck 104 and a peripheral. Thenumber of connection points within a receptacle is dependent on thenumber of connection points required for the underlying protocol used bythe connectors 330A-330C. For example, UBS-C® connector would include 10connection points whereas a LIGHTNING® connection point would includeeight connection points. In one embodiment, the housing of connectors330A-330C may comprise a plastic-injection molded housing.

Each connector 330A-330C includes a set of four hinge elements (e.g.,332A-1, 332A-2, 332A-3, and 332A-4). Each of these hinge elementsextends outwardly away from the housing. In the illustrated embodiment,the hinge elements are curved according to the circumference of the band322. In the illustrated embodiment, the hinge elements may be formed aspart of the housing (e.g., via injection molding or via attachment).Thus, the hinge elements and the housing can in some embodiments form asingle contiguous element. In one alternate embodiment, the hingeelements are suitably flexible to allow the connectors 330A-330C to“snap” onto the band 332. In one embodiment, a user of the puck 104snaps the connectors 330A-330C onto the band by situating the connectors330A-330C against the band (with the hinge elements facing the band) andexerts an inward force to attach the connectors 330A-330C to the band322. The coefficient of friction between connectors 330A-330C and band332 may be designed such that each connector 330A-330C may be freelymoved along band 332, thus allowing each connector 330A-330C to bepositioned at any location around the band 332. Additionally, eachconnector 330A-330C may be rotatable toward the top side and bottom sideof the puck 104. In one embodiment, the coefficient of friction may bedesigned such that the upward or downward rotation of the connectors330A-330C causes the connectors 330A-330C to be fixed at an upward ordownward angle.

In one embodiment, each connector 330A-330C includes a plurality ofconductive pads positioned on the underside of the housing, that is,facing the band 322 when connected. Specifically, these conductive padsare positioned opposite the receptacles 333A-333C. As described above,the number of these conductive pads is dependent on the protocol usedfor the connectors 330A-330C. Similar to the embodiments discussedabove, the band 322 includes a plurality of conductive traces equal toor greater than the number of connective pads on the connectors330A-330C. Thus, when snapped onto the band 322, each connector330A-330C is communicatively coupled to the puck 104 via the conductivetraces on the band 322 and the conductive pads on the connectors330A-330C. In one embodiment, the conductive pads may be alignedvertically to match the position of the traces. In other embodiments theconductive pads may be spaced both vertically and horizontally on thehousing so long as a single conductive pad is in contact with a trace.For example, the conductive pads may be positioned along a diagonal lineon the underside of the connectors 330A-330C.

In some embodiments, the conductive pads and traces may be greater thanthe number of connection points of the protocol used by the connectors330A-330C. For example, one or more additional control traces and padsmay be installed on the band 322 and connectors 330A-330C respectively.In one embodiment, these additional connection points may be used tocontrol the location of a given connector 330A-330C. In one embodiment,the band 322 and connectors 330A-330C may additionally include amagnetized or detectable control path enabling the connectors 330A-330Cto be repositioned programmatically. For example, movable magnets (e.g.,micro-electrical mechanical motors attached with or without magnets, ora worm gear drive or screw/gear or other linear actuator) may beconfigured to rotate around band 322 (e.g., via motor control ormanually). In one embodiment, these magnets or position controllers maybe situated on the top side of band 322. Each connector 330A-330C mayadditionally include magnetic or otherwise engageable portion on the topside of the housing that is in contact with the band 322. Upon movingthe movable magnet or other actuator mechanism within or on the band322, a respective connector 330A-330C moves in synchrony, thus allowingthe connectors 330A-330C to be rotated around the band 322.

FIGS. 4A-4H depict a modular headphone-based VR/AR device with adetachable goggle portion according to some embodiments of thedisclosure.

As illustrated in FIG. 4A, and discussed previously, goggle portion 110can be fixedly connected to ear portion 120 via a detachable connector.Additionally, in some embodiments, the detachable connector canadditionally include a USB-C® connection allowing for datacommunications between headband 102 and puck 104.

As illustrated in FIGS. 4B-4E, goggle portion 110 can be removed fromear piece portion 120. In this configuration, pucks 104 a, b andheadband 102 can continue to operate despite the removal of the goggleportion 110. In some embodiments, the device 100 can continue to providefunctionality such as audio playback, light capture, and position orhead tracking (light capture and position or head tracking are describedmore fully in connection with FIGS. 6A through 6C). In some embodiments,device 100 can be configured to transmit data regarding position or headmovements to external devices while in a detached state. For example,device 100 can be configured to transmit position or head movements to aset top box while a user is watching television, or a detached puck orpucks can act as a set top box to control one or more video displays, orthe puck itself can act as a set top box and via the pico projectorsgenerate a video or VR/AR space within which the user wearing thepuckless or single-pucked device 100 moves or interacts. By transmittingposition or head movements, a device 100 can be configured to act as aninput device for the set top box. In some embodiments, the set top boxcan be configured to receive position or head movements to monitor userinteraction with the set top box.

FIG. 4F illustrates a VR/AR device. As illustrated in FIG. 4F, goggleportion 110 can be connected to ear piece portion 120 via arm 106. Inone embodiment, the connection between ear piece portion 120 and arm 106can comprise a rotatable joint or circular hinge that allows goggleportion 110 to be rotated up toward headband 102.

FIG. 4G illustrates a VR/AR device including a drone hook. Asillustrated in FIG. 4G, and discussed previously, drone hook 312 can bepulled upward, away from puck 104 to allow for removal by a drone (asdiscussed more fully in connection with FIG. 7) or used as a kick-stand.

FIG. 4H illustrates an adjustment position for a goggle portion of aVR/AR device. As illustrated, in a first state 410 goggle portion 110can be positioned at a first angle for viewing a VR/AR scene. In state412, goggle portion 110 can be moved by moving arm 106. In someembodiments, a user can move goggle portion 110 toward or away from auser's eyes, or up or down relative to a user's eyes, to adjust theviewing angle of the VR/AR device while in use to suit a user's faceshape, eye position, accommodate prescription glasses worn by a user,and other comfort based positioning. In some embodiments, arm 106 can beconnected to ear piece portion 120 via a rotatable connector allowingfor relative rotative movement of the goggle portion 110 as describedabove.

FIG. 5 is a block diagram illustrating a puck or headband mountedaccessory according to some embodiments of the disclosure.

As illustrated in FIG. 5, a puck 500 includes processor(s) 502, memory504, input device(s) 506, co-processor(s) 508, a display 510, one ormore network interface cards 512, USB-C® connector 514, power supply516, audio interfaces 518, and keypad 522. Although illustrated asseparate components, in some embodiments, some or all components of puck500 can be implemented as a SoC design and thus can be tightlyintegrated.

In the illustrated embodiment, puck 500 includes one or more processors502. In one embodiment, processors 502 can be utilized to control theoperation of the puck 500 during contemplated operations. For example,processors 502 can be utilized to coordinate access to input/outputdevices and display devices, access to memory, and other general purposefunctions. In one embodiment, processors 502 can be a fully realizedmobile processor such as that sold as the Qualcomm 835 SNAPDRAGON®, orthe NVIDIA TEGRA® X1 processors or similar processors. Such processorsare known in the art to incorporate multiple functional components to:receive environment data (video, audio, position, motion, orientationand the like) from multiple sensor types; contain modems forfacilitating the receiving and transmitting data to and from otherdevices using multiple communication protocols; process and render highspeed graphics with graphic processing units (GPUs); perform digitalsignal processing using a digital signal processor (DSP); perform rapidimage capture using an image sensor processor (ISP); drive multiplevideo devices with up to 4k resolution; contain programmablemicroprocessor(s) and memory, all in a fan-less single chip structure.Thus a puck or headband mounted accessory utilizing this type ofprocessor to integrate the multiple elements described herein can act asa full function mobile device or set top box or smart TV as well as afull function VR/AR device, yet be easily worn on a headphone assemblyfor exceptional comfort, wear-ability, ease of long use (due to comfortand/or battery life), and free interchangeability on the assembly 100 ina manner heretofore unrealizable in the absence of the flexible assemblystructure and methodology disclosed herein. Moreover these processorsare becoming ubiquitous in the market and the ease of connectingperipheral devices to interact with such processors is becoming acceptedby persons of skill, who will readily recognize the benefits of themodular device concept introduced in this disclosure. In someembodiments, processors 502 can be implemented as an ASIC. In oneembodiment, processors 502 can include one or more graphics processingunits (“GPUs”).

For example a user may start with a puck or pucks utilizing a lower costlower function processor chip, and then upgrade to a puck containing ahigher cost higher function processor, without having to abandon theheadphone assembly 100, thus providing consumers a continuous pathway tohigher functions in a modular manner without the risk of rapidobsolescence.

In some embodiments, processors 502 can be selected based on heatingrequirements of the puck 500. For example, processors 502 can beselected to obviate the use of a fan or mechanical cooling system.

Processors 502 can be utilized to manage communication along bus 522.Processors 502 can additionally be configured to control access to otherdevices on the bus 522 in response to requests received via USB-C®connector 514. As discussed herein, puck 500 can be connected via USB-C®connector 514 to other pucks via a bus. In this embodiment, processors502 can receive requests for data and/or processing operations fromother pucks and can coordinate external processing requests from otherpucks. For example, another puck can transfer a process to puck 500 forprocessing. In response, processor 502 can accept or deny the externalprocess and, if accepting, can schedule processing of the process. Insome embodiments, scheduling an external process can comprisecoordinating access to co-processors 508.

As discussed, puck 500 can include one or more co-processors 508. In oneembodiment, puck 500 can be configured to perform specific VR or ARoperations. In these embodiments, puck 500 can be equipped withspecialized hardware to perform these operations. In some embodiments,co-processors can comprise an NVIDIA TEGRA® X1 processor, Qualcomm 835SNAPDRAGON® processor, or a MICROSOFT® HPU. In some embodiments,co-processors 508 can be selected based on compatibility with the one ormore processors 502, or can act fully as processor 502. In someembodiments, co-processors 508 can be implemented as afield-programmable gate array (FPGA).

As an example, puck 500 can include additional co-processors to performposition-based tracking and/or light field capturing. Examples ofspecific configurations of co-processors 508, and associated peripheraldevices, are described more fully in connection with FIGS. 6A through6C. Although illustrated as processor devices, processors orco-processors can comprise a SoC or FPGA device and can additionallyinclude further input and/or output devices as described in connectionwith FIGS. 6A through 6C. In some embodiments, processors 502 andco-processors 508 can be combined into a single chip or SoC.

Power supply 516 provides power to the components of puck 500. Arechargeable battery can be used to provide power. In one embodiment,power supply can be recharged via a USB-C® input or via an inductivecharging pad or other wireless charging methodology. In one embodiment,power supply 516 can be configured to be recharged using a drone-basedrecharging system as discussed more fully in connection with FIGS. 7-8B.In some embodiments, power supply 516 can comprise a batterysignificantly larger than existing VR or AR devices due to its placementon the side of the head or in the headband versus on the front of a VRor AR device as in current goggles and masks. In some embodiments, powersupply 516 can be connected to an inductive power charging pad or deviceas described more fully herein.

Network interfaces 512 include circuitry for coupling puck 500 to one ormore networks, and are constructed for use with one or morecommunication protocols and technologies. Network interfaces 512 aresometimes known as a transceivers or transceiving devices. In oneembodiment, network interfaces 512 include Wi-Fi, Bluetooth, cellular,or NFC interfaces. In some embodiments, puck 500 can be configured toreceive three-dimensional or augmented reality data streaming from aremote server for display using device 100.

Audio/video interfaces 518 are arranged to produce and receive audiosignals and video signals. In one embodiment, audio/video interfaces 517can receive audio data such as the sound of a human voice or audioassociated with a VR or AR scene. For example, audio/video interfaces518 can be coupled to a speaker and microphone to enabletelecommunication with others and/or generate an audio acknowledgementor user voice commands to initiate some action. Alternatively, or inconjunction with the foregoing, audio/video interfaces 518 can receivevideo data for further processing. For example, puck 500 can be equippedwith one or more cameras configured to record video and display videovia display 510. Additional video interfaces (and supporting hardware)for capturing light field images are described more fully in connectionwith FIGS. 6A through 6C.

Display 510 can be a liquid crystal display (LCD), gas plasma, or OLEDdisplay, or any other type of display used with a computing device.Display 510 can also include a touch sensitive screen arranged toreceive input from an object such as a stylus or a digit from a humanhand. Display 510 can also be a pico projector as discussed herein.

Keypad 520 can comprise any input device arranged to receive input froma user. For example, keypad 520 can include a push button numeric dial,or a keyboard. Keypad 520 can also include command buttons that areassociated with moving backward and forward through one or more userinterfaces. In one embodiment, keypad 520 can comprise an expandableaccordion keyboard that can be retractable from puck 500.

Puck 500 also comprises input/output interface 506 for communicatingwith external devices, or other input or devices not shown in FIG. 5.Input/output interfaces 506 can utilize one or more communicationtechnologies, such as USB, infrared, Bluetooth, Wi-Fi or the like.

Memory 504 can include a RAM, a ROM, and other storage means. Memory 504illustrates another example of computer storage media for storage ofinformation such as computer readable instructions, data structures,program modules or other data. Memory 504 stores a basic input/outputsystem (“BIOS”) for controlling low-level operation of puck 500. Thememory 504 can also store an operating system for controlling theoperation of puck 500. It will be appreciated that this component caninclude a general purpose operating system such as a version of UNIX, orLINUX™, or a specialized client communication operating system such asWINDOWS CLIENT™, or the SYMBIAN® operating system. The operating systemcan include, or interface with a Java virtual machine module thatenables control of hardware components and/or operating systemoperations via Java application programs.

Puck 500 additionally includes a USB-C® connection 514. As discussedpreviously, puck 500 can be connected to a headband or other apparatusvia the USB-C® connection 514. Puck 500 can additionally be configuredto transmit VR or AR data to an external display device (e.g., a set ofVR goggles or pico projector(s) via USB-C® connection 514 or wirelessly.Additionally, USB-C® connection 514 can be used to charge the device.

FIG. 6A is a block diagram illustrating internal components of a puck orheadband mounted accessory according to some embodiments of thedisclosure (which may also incorporate or be incorporated intocomponents depicted in FIG. 5).

As discussed previously, pucks can be configured in a modular manner soas to provide different functionality or functionalities as needed bythe VR/AR device. In the illustrated embodiment, puck 600 a isconfigured to allow for both positional tracking of a VR/AR device aswell as light capture. In alternative embodiments, puck 600 a caninclude additional components to provide additional functionality asneeded, including functionality as discussed previously.

In the illustrated embodiment, puck 600 a can include a photosensorarray 602 that includes a plurality of photosensors 602 a-n. In theillustrated embodiment, photosensors 602 a-n can be placed on theexternal surface of the puck 600 a as illustrated, for example, in FIGS.1A through 1D and 3A through 3R.

In the illustrated embodiment, photosensors 602 a-n can be utilized withone or more “outside-in” positional tracking devices. In an outside-inpositional tracking system, a user can utilize one or more positionaltracking projectors which provide reference points that allow a deviceto determine the position of a user within a physical space. In oneembodiment, a light-based positional projection device can be utilizedto implement positional tracking as described herein.

In some embodiments, puck 600 a can be configured as a LIGHTHOUSE™tracking device utilized as part of a LIGHTHOUSE™ tracking systemprovided by Valve Corporation. In this embodiment, one or moreLIGHTHOUSE™ reference points can be placed within a physical space andphotosensors 602 a-n can be configured to detect pulses of light andlaser sweeps throughout the physical space to detect the position of thepuck. In alternative embodiments, puck 600 a may not utilize outside-intracking. In these embodiments, photosensor array 602 can be omittedfrom puck 600 a and rivets associated with the photosensor array canlikewise be omitted.

In the illustrated embodiment, photosensors 602 a-n can be configured todetect light emitted from a positional projection device (notillustrated) at periodic intervals. In one embodiment, a positionalprojection device can comprise a light emitting device that periodicallyoutputs light from one or more light sources and “fills” a physicalspace with light at regular, defined intervals.

In the illustrated embodiment, puck 600 a can store an identifier foreach photosensor 602 a-n representing its location on the exterior ofpuck 600 a. Upon detecting a light source, the photosensors 602 a-ntransmit signals to position tracking processor 604 indicating that alight source was detected by one or more of the photosensors 602 a-n.Position tracking processor 604 can initiate a timer and record andprocess the timing of received signals from photosensors 602 a-n andthus be capable of determining a user's position relative to thepositional projection device(s).

Position tracking processor 604 can transmit the detected position ofthe puck (and thus, user) to scene renderer 606. As illustrated in FIG.6A, scene renderer 606 can comprise one or more processors configured toretrieve scene data from memory 608. For example, memory 608 can includeone or more three-dimensional scenes to be displayed on device 610 (orprojected using the pico projectors). In some embodiments, individualscenes may be linked together to form a “map” of a physical space. Byreceiving a user's position, scene renderer 606 can identify a virtualrepresentation of a scene to be rendered based on the scene data storedin memory 608. In some embodiments, scene renderer 606 can include oneor more modules to provide the geometry which the scene renderer 606combines with the animation, textures and the like to output a renderedscene comprising the virtual representation. In the case of alight-field or reflectance field, the scene renderer 606 uses theaforementioned data to appropriately update the display. In someembodiments, scene renderer 606 can comprise or utilize one or moreNVIDIA TEGRA® X1 processors, Qualcomm SNAPDRAGON® processors, or aMicrosoft Holographic Processing Unit (HPU) processor(s). Additionally,in some embodiments scene renderer 606 can comprise a SoC containingcomponents to carry out the functions described herein.

In addition to receiving a user's position from position trackingprocessor 604, scene renderer 606 can receive head position informationfrom head tracking subsystem 612. In one embodiment, head trackingsubsystem 612 includes an accelerometer 612 a, gyroscope 612 b,magnetometer 612 c. In the illustrated embodiment, the head trackingsubsystem 612 transmits head tracking data associated with the user'shead (as measured by accelerometer 612 a, gyroscope 612 b, magnetometer612 c) to scene renderer 606. In response, scene renderer 606 adjusts arendering of the virtual representation based on the head tracking data.In some embodiments, position tracking processor 604 can comprise one ormore NVIDIA TEGRA® X1 processors or Qualcomm SNAPDRAGON® processors.Additionally, in some embodiments, scene renderer 606 can comprise aSoC. In some embodiments, position tracking processor 604 and scenerenderer 606 can be combined in a single SoC chip design. Alternatively,position tracking processor 604 and scene renderer 606 can beimplemented as a single processor.

Additionally, as illustrated in FIG. 6A, puck 600 a can include a lightfield capturing array 616 including capture devices 616 a-n. In oneembodiment, a capture device can comprise a light source and cameradevice situated at positions on the exterior of puck 600 a.Alternatively, capture array 616 can include separate light sources andcameras as capture devices 616 a-n. In some embodiments, capture array616 can only include light sources or cameras, as discussed previously.

In the illustrated embodiment, light field capturing array 616 can beconfigured to capture multiple images of a three-dimensional space underdiffering lighting conditions. In some embodiments, the images cancomprise static photographs or video. In some embodiments, the cameradevices can be fitted with wide angle lenses or fisheye lenses. In someembodiments, the cameras can be placed within recesses or spaces formedon the exterior of puck 600 a.

In some embodiments, the camera devices can comprise light field camerasor plenoptic cameras. A light-field camera aims to measure the intensityand direction of every incoming ray instead of merely recording the sumof all the light rays falling on each photosite at a sensor. With suchinformation every possible image of whatever is within the field of viewof the camera at the moment of image capture can be generated. A singlecapture from a light-field camera can provide digital data such thatfocus, exposure and even depth of field are adjustable after the imageis captured.

Capture devices 616 a-n, under the control of capture processor 614, canbe configured to capture a plurality of images of a three-dimensionalspace and transmit the captured images to capture processor 614. In someembodiments, each camera can be capable of being adjusted (e.g., focus,angle, etc.) by capture processor 614. In some embodiments, captureprocessor 614 can receive adjustment commands from a user via an inputdevice.

In one embodiment, the field of view of the cameras is very widerelative to the spacing of the horizontal and vertical viewpoints, suchthat most of one image captured by one of the cameras covers the samescene content as the ones captured by the adjacent cameras, though froma slightly different viewpoint. The images thus obtained from thecameras in the light field capturing array 616 do not have to bestitched into a single panorama, but rather can be stored in a databaseas a two-dimensional array of two-dimensional images, forming afour-dimensional light field dataset.

During operation, each of the cameras in capture devices 616 a-n can beactivated by the capture processor 614 and are activated simultaneouslyto collect image data. In one embodiment, a puck 600 a can be placed onboth sides of a user's head as illustrated in FIGS. 1A-1D. In thisembodiment, a near-360 degree view of a scene can be recorded.

In addition to camera devices, capture devices 616 a-n include lightsources. In one embodiment, capture processor 614 can be configured toactivate the light sources in conjunction with camera devices in orderto illuminate a subject or space with a sequence of time-multiplexedlighting configurations. In one embodiment, the capture processor 614can be configured to adjust the direction of the light sources in orderto focus the light on the subject or space being imaged at a particulartime and at a specific angle or at a specific polarization.

In one embodiment, a light source can comprise a light-emitting diode(“LED”). In some embodiments, alternative light sources can be used inplace of LED devices. Alternatively, or in conjunction with theforegoing, the light sources used as part of capture devices 616 a-n cancomprise polarized light sources.

In alternative embodiments, puck 600 a can additionally utilize camerasin the capture array 616 to measure the depth of an area. In thisembodiment, some or all of the cameras in capture array 616 can comprisedepth cameras. In this embodiment, depth cameras can be utilized inplace of, or in conjunction with, photosensors 602 a-n to calculate auser's position and changes in positions in a physical space in an“inside-out” manner.

Although discussed as a single puck above, a VR/AR device can beconfigured with multiple pucks and/or headband mounted accessories asdiscussed previously. For example, a VR/AR device can include two pucksincluding the components discussed in connection with puck 600 a. Inthis embodiment, the two pucks can be communicatively coupled to eachother via a bus spanning a headband, as illustrated in FIGS. 1A-1D, orvia a wireless connection. In this arrangement, each puck would beconnected via a USB-C® (or wireless) connection spanning the headbandand may work cooperatively with each other and/or with a similarlyconfigured headband mounted accessory to perform the operationsdiscussed in connection with puck 600 a. For example, both pucks and/orheadband unit can be configured with a capture array and a positiontracking processor. In this arrangement, the position tracking processorof each puck can coordinate the position tracking operations discussedabove. For example, the position tracking processors in each cantransfer data between the pucks and/or headband unit to distributeprocessing loads. Alternatively, or in conjunction with the foregoing,each puck can be equipped with a capture array and a capture processorand can distribute processing loads evenly between the pucks. Suchtransfers are beneficial in the event that one puck detects a minimalworkload due to the positioning of the puck. In alternative embodiments,a first puck can be configured only to perform position trackingoperations while the other puck can be configured to perform lightcapture operations.

As discussed previously, puck 600 a (or headband accessory) can includeone or more batteries. As puck 600 a includes one or more batteries, theaforementioned operations may be performed by puck 600 a even while puck600 a is detached from a headband as illustrated in FIGS. 1A-D. Forexample, a user may detach puck 600 a from a headband and utilize thepuck as a light capture apparatus to capture a three-dimensional sceneincluding a user wearing a VR/AR device. In this embodiment, puck 600 acan further include one or more network interface for transmitting athree-dimensional scene to additional users.

In some embodiments, position tracking processor 604 and captureprocessor 614 can be implemented in an FPGA. In this embodiment, scenerenderer 606 can be implemented via a multi-purpose processor (like anSoC) or GPU or general purpose processor. In some embodiments, scenerenderer 606, head tracking subsystem 612, and memory 608 may all beimplemented in a multi-purpose purpose processor or SoC. Theaforementioned embodiments are discussed more fully in connection withFIGS. 6B and 6C.

FIG. 6B is a circuit diagram illustrating internal components of a puckor headband mounted accessory according to some embodiments of thedisclosure (which may also incorporate or be incorporated intocomponents depicted in FIG. 5).

FIG. 6B illustrates a distributed architecture for providing thefunctionality of a puck. As illustrated, the architecture 600 b isdivided into a shared processor section that includes a CPU 618 and anFPGA-based section that includes one or more FPGAs 620, LEDs 622, andphotodiodes 624.

As illustrated in FIG. 6B, the FPGA 620 is utilized to drive thephotodiodes 624 and LEDs 622. In one embodiment, FPGA 620 is utilized tocontrol the operation photodiodes 624 and LEDs 622 as well as receiveand route data from photodiodes 624 and LEDs 622 to CPU 618.

CPU 618 can comprise a multi-purpose processor such as a Qualcomm 835SNAPDRAGON® or NVIDIA TEGRA® X1 processor, sometimes referred to assystems on a chip (SoCs). CPU 618 communicates with FPGA 620 via sharedprocessor interface (“SPI”) 628. In one embodiment, SPI 628 comprises aUSB-C® interface. In one embodiment, the shared processor section islocated in a removable housing and can be connected and disconnectedfrom the FPGA-based section.

FIG. 6C is a circuit diagram illustrating internal components of a puckor headband mounted accessory according to some embodiments of thedisclosure (which may also incorporate or be incorporated intocomponents depicted in FIG. 5).

FIG. 6C illustrates a distributed architecture for providing thefunctionality of a puck. As illustrated, the architecture 600 c isdivided into an FPGA-based section that includes one or more FPGAs 620,LEDs 622, and photodiodes 624 and a shared processor section thatincludes a CPU 618. The two sections can be connected via SPI bus 626.SPI bus 626, CPU 618, FPGAs 620, LEDs 622, and photodiodes 624 aredescribed in connection with FIG. 6B.

Additionally, illustrated in FIG. 3C are cameras 628. In embodiments,cameras 628 comprise cross-polarized stereo camera pair modules and,optionally, depth sensors. In some embodiments, a cross-polarized stereocamera pair module can comprise multiple discrete cameras including awide-angle fish-eye camera and adjacent cameras. In some embodiments,projection devices can optionally include cross-polarized stereo camerapair modules and optional depth sensors. Cameras 628 are connected toCPU 618 (which can be an SOC) via a MIPI (“Mobile Industry ProcessorInterface) interface 630 although alternative interfaces may beutilized.

FPGA 620 is connected to cameras 628 via a synchronization bus 632. Insome embodiments, synchronization bus 632 can comprise a USB-C® bus orother signal path. In alternative embodiments, FPGA 620 can synchronizecameras 628 through CPU 618. Synchronization bus 632 allows for thesynchronization of LEDs 622 and cameras 628 during light captureoperations as discussed previously, as well as other cross-functionsyncing functions.

CPU 618 is additionally communicatively coupled to projector 634 via abus, such as HDMI (High-Definition Multimedia Interface) bus 636. Insome embodiments, CPU 618 can be connected to multiple projectors. Asdiscussed previously, projector 634 can comprise a swivelable orpositionable pico projector. In some embodiments, projector 634 caninclude an RGBZ module. As discussed previously, projector 634 can beconfigured to project light field scenes or other visible video or imageinformation into a physical space, generally onto one or more surfacesin the space.

CPU 618 receives video streams from one or more interfaces 638, 640,642. As illustrated, CPU 618 can receive a video stream via a USB-C®interface 638. In one embodiment, video streams from other pucks or froma headband portion of a VR/AR device, described previously, can providevideo streams to CPU 618 via UBS-C interface 638. Video streams can alsobe transmitted to CPU 618 via an IEEE 802.11AD interface 640. Inalternative embodiments, other Wi-Fi interfaces can be used to transmitvideo streams to CPU 618. Video streams can also be transmitted to CPU618 via a LTE (Long-Term Evolution) interface 640. In alternativeembodiments, other cellular interfaces can be used to transmit videostreams to CPU 618. In alternative embodiments, video streams can bereceived by CPU 618 via satellite interfaces and/or radio interfaces.Although illustrated outside the shared processor section, in someembodiments, interfaces 638, 640, and 642 can be implemented as part ofCPU 618. As illustrated, CPU 618 can receive video stream data from aVR/AR device itself (e.g., via USB-C® interface 628) or from a remotedata source via 802.11AD interface 640 and/or LTE interface 642. Forexample, CPU 618 can receive three-dimensional scenes transmitted from aremote rendering site (e.g., a cloud rendering application or service).

FIG. 7 is a block diagram illustrating a drone-based recharging systemaccording to some embodiments of the disclosure.

As illustrated in FIG. 7, one or more wired charging devices 702 a-n canbe placed throughout a physical space. In one embodiment, a chargingdevice can comprise an electronic device designed to be inserted into apower outlet such as wall-mounted power outlet. In alternativeembodiments, charging device can comprise an electronic device connectedto another electronic device (e.g., a laptop, desktop, etc.) via a USBor similar connection. In each case, the charging device can beconfigured to draw power from a fixed power source. Charging device caninclude power circuitry known in the art to perform standard poweroperations.

Charging devices 702 a-n additionally includes an external chargingportions 704 a-n. In some embodiments, a charging portion 704 a cancomprise an inductive charging surface. In alternative embodiments, acharging portion 704 a can comprise a USB port.

As illustrated, a drone device 706 can be capable of connecting a puck710 to charging portion 704 n. Alternatively, or in conjunction with theforegoing, drone device 706 can include a rechargeable battery capableof being recharged via charging portions 704 a-n. In one embodiment,drone device 706 can place a puck on charging device 702 n until neededby VR/AR device 708, the operation of which is described below.

As discussed previously, pucks connected to VR/AR device 708 include arechargeable battery. In some embodiments, the pucks can monitor theiravailable battery level and can transmit a notification to one or moredrone devices 706 upon detecting that the available battery level hasreached a limiting threshold. For example, a puck 710 can monitoravailable battery life and notify drone device 706 upon detecting aremaining battery level of 10%.

Drone device 706 can include one or more network interfaces designed toreceive notification from the pucks. In one embodiment, drone device 706can include Bluetooth, Wi-Fi, NFC, or other radio-based networkinterfaces to allow for communication with a VR/AR device 708.

Upon detecting a notification from VR/AR device 708, one or more dronedevices (e.g., 706) can activate and detach a charged puck (e.g., 710 b)from charging device 702 a. As described in connection with FIGS. 8A and8B, a drone device 706 can comprise a quadrotor type drone. Afterdetaching a puck 710 from charging device 702 a, drone device 706 cannavigate to the VR/AR device 708. In one embodiment, drone device 706can utilize one or more LED lights present on the VR/AR device 708and/or pucks to identify a location of the VR/AR device 708 and navigateto the VR/AR device 708.

Upon reaching VR/AR device 708, drone device 706 can position itselfabove ear piece portion and lower hook 712 using motor 716. Asillustrated in FIG. 7 and discussed previously, puck 710 can include aretractable drone hook 718 in a recessed state when connected to VR/ARdevice 708. Drone device 706 can be configured lower hook 712 such thata curved portion of hook 712 engages drone hook 718.

Once the drone device 706 attaches the hook 712 to drone hook 718, thedrone device 706 can move outward and/or upward from the user in orderto disconnect the puck 710 from VR/AR device 708.

Drone device 706 can then return the puck 710 to available chargingdevice 702 n. As illustrated, drone device 706 can be configured toplace the puck 710 on an inductive charging pad of charging device 702n. Once placed on charging pad 704 n, drone device 706 can wait untilthe puck 710 is fully charged. In some embodiments, puck 710 cancontinue to broadcast a battery state while charging to notify dronedevice 706 of the charge state of puck 710.

In some embodiments, multiple charging devices 702 a-n can be utilized.In this embodiment, drone device 706 can identify a fully charged puckplaced on another charging device (e.g., 702 a). Upon detecting a fullycharged puck, the drone device 706 navigate to the fully charged puck,pick up the puck 710 b using hook 712 and delivery the fully chargedpuck to VR/AR device 708. This, the system can allow for continuedoperation of one or more pucks by swapping pucks based on battery lifenotifications.

FIGS. 8A and 8B illustrate drone delivery devices according to someembodiments of the disclosure.

As illustrated in FIGS. 8A and 8B, drone device 800 can comprise aquadrotor type drone. It may be appreciated that FIGS. 8A and 8B andrelated description is only provided only by the way of illustration andnot limitation and that any type of aerial or multi-mode vehicle ofsuitable attributes currently known or to be invented can be employed.The drone 800 includes an attachment portion 802 located underneath anupper frame 804. The upper frame 804 operates to hold shafts 808 a-dfixed to the attachment portion 802 in an ‘X’ shaped configuration. Hub810 can be connected to rotor shaft apparatus 812. As illustrated inmore detail in FIG. 8B, rotor shaft apparatus 812 can comprise arotatable ‘X’-shaped component. At the end of each arm of rotor shaft812 are rotors 814 a-d. Rotors 814 a-d can comprise two pairs ofcounter-rotating, fixed-pitch blades located at the four corners of thevehicle. The attachment portion 802 can also comprise a power source topower the vehicle and its onboard circuitry along with any computerreadable storage media.

As illustrated, rotor shaft apparatus 812 is coupled to a motor 816which includes a retractable chain 818. In some embodiments, chain 818can comprise any suitably flexible material capable of being lowered andraised by motor 816. Chain 818 connects the motor 816 to hook 820. Inone embodiment, hook 820 can comprise a spindle top portion 822connected to chain 818, a vertical hook shaft 824, and a curved hook end826. Hook end 826 can be configured to connect to drone hook 312connected to puck 104, as described previously. In some embodiments,drone device 800 can be fitted with imaging equipped such as cameras andphotosensors in order to detect the location of puck 104 as describedpreviously.

As illustrated and discussed previously, drone 800 can be configured tofly to a VR/AR device and attach hook end 826 to the drone hook 312 of apuck 104 that is in use by the VR/AR device. In this manner, drone 800can be capable of replacing pucks of a VR/AR device as needed and asdiscussed in connection with FIG. 7. Additionally, drone 800 can beconfigured to fly with puck 104 attached to perform light captureoperations as described more fully in connection with FIG. 9.

FIG. 9 is a block diagram of a drone-based capture system according tosome embodiments of the disclosure.

As illustrated in FIG. 9, a capture system 900 can include a pluralityof drones 902 a-n. In some embodiments, drones 902 a-n can comprisequadrotor type drones equipped with sensor and imaging equipment thatcan be flown into a wide variety of patterns or positions or paths tofacilitate image data capture. In some embodiments, imaging equipmentcan comprise light sources (e.g., LEDs), cameras, optical sensors,infrared sensors, radio sensors, polarized light sources, or acombination thereof. The structure and operation of drones 902 a-n isdescribed more fully in connection with FIGS. 8A and 8B.

In the illustrated embodiment, drones 900 a-n can be equipped withcommunications components to allow for communication with VR/AR device100, and in particular, puck 104. In one embodiment, drones 900 a-n canbe equipped with Bluetooth, Wi-Fi, or other radio-based transceivers andcan be configured to receive commands from puck 104. Alternatively, orin conjunction with the foregoing, drones 900 a-n can additionally beconfigured to transmit image or other data to puck 104 for furtherprocessing and/or display.

In some embodiments, puck 104 can be utilized to control the movement ofdrones 902 a-n. In some embodiments, drones 900 a-n can be configured totrack the position of the VR/AR device 100 while in flight and can beconfigured to fly in a particular pattern. In one embodiment, a patterncan be programmed by a user and/or via the device 100. In oneembodiment, a pattern comprises a plurality of points representingpositions in three dimensional space having therewithin a subject.Drones 902 a-n are positioned at points to form the pattern in the spacein proximity to the device 100 (and/or a pre-defined image subject). Atvarious times, drones 902 a-n can activate imaging components on puckscarried by drones 90 a-n for collecting imaging data of the subject orarea. In some embodiments, pucks carried by drones 902 a-n can beconfigured to capture and render three-dimensional light field images asdiscussed previously. In this embodiment, drones 902 a-n can beconfigured to transmit the rendered scene data to puck 104 for displayon VR/AR device 100.

Each of the drones 902 a-n can be configured to fly independently andcan have a respective trajectory mapped out to reach and levitate at aparticular position or continuously or repeatedly follow a certaintrajectory in space. However, they can also be remotely controlled ortheir flight paths can be monitored and altered by a device 100.

The drones 902 a-n are arranged in a particular pattern with respect toa subject being imaged. It may be appreciated that the number andpattern of the drones 902 a-n is shown only by the way of illustrationand that greater or lesser number of drones 902 a-n can be used invarious patterns to generate different lighting conditions or to collectimage data from various positions or angles as will be detailed furtherherein.

In one embodiment, each of the drones 902 a-n can receive respective,unique position information from the device 100, map a flight path ortrajectory in order to reach a respective, designated position at apredetermined time. The position information for each drone can begenerated based on one or more of a selected arrangement for theplurality of drones 902 a-n, or attributes of the drones 902 a-n as willbe detailed further herein. In one embodiment, the device 100 and/or thesubject in combination with the ground or base surface can be used bythe drones 902 a-n as a reference entity to achieve their respectivepositions. In one embodiment, one or many radio sources may be presentat the location of the subject so that the drones 902 a-n are able toidentify the subject and therefore position themselves accordingly in adesignated pattern in proximity to the subject. Alternatively, or inconjunction with the foregoing, device 100 (or pucks thereon) can beused as a radio source. This can be useful in external or outdoorenvironments where there may be a multitude of objects and particularidentification of the subject to be imaged may be required.Alternatively, sonar can be used (either on the drone itself or using anexternal sonar device to track the drone) to provide positioninformation to one or more of the drones 902 a-n relative to the subjector other object of known position or other drones 902 a-n.

The subject or object being imaged can be illuminated with the lightfrom the pucks held by drones 902 a-n. In one embodiment, all the pucksneed not emit light. In an embodiment, the pucks can be pre-programmedto activate the LEDs and illuminate the subject with a sequence oftime-multiplexed lighting configurations. The light sources on pucks canbe selectively activated to emit light at a particular time after thedrones 902 a-n reach their designated positions. Image data of thesubject thus illuminated can be recorded by one or more cameras on thepuck. In one embodiment, the camera(s) can also be controlled by thedevice 100 when the drones 902 a-n have achieved the desired formationand illuminate the subject in a desired manner. In an embodiment, thefunctioning of the camera(s) and the plurality of drones 902 a-n can besynchronized such that the camera(s) automatically capture the imagedata upon the drones 902 a-n achieving particular configurations. As thedrones 902 a-n are small and wireless, they are portable and may becarried easily to a location of a subject. As described previously, thepucks carried by drones 902 a-n can comprise different types of lights,cameras, filters, sensors or combinations thereof. Hence, in contrast tothe efforts and time utilized in adjusting conventional lighting andcamera equipment, the illustrated system affords simple adjustmentswherein one or more of the pucks can be swapped with other differenttype(s) of pucks in order to produce a different lighting effect orrecord different type of image data as needed. As described previously,pucks can be equipped with light-field cameras that measure theintensity and direction of every incoming ray instead of merelyrecording the sum of all the light rays falling on each photosite at asensor. With such information every possible image of whatever is withinthe field of view of the camera at the moment of image capture can begenerated. A single capture from a light-field camera can providedigital data such that focus, exposure and even depth of field areadjustable after the image is captured.

In one embodiment, device 100 can instruct drones 902 a-n to positionthemselves within a three dimensional space in a selected pattern orgeometry. The pattern for arranging the drones 902 a-n can be selecteddepending on various factors including but not limited to, the size ofthe subject being imaged, the nature of the surface/subject beingimaged, the kind of image data necessary and the attributes of thedrones 902 a-n and/or pucks 104 a-c. At least a subset of the pluralityof drones 902 a-n is selected for activation. The pucks 104 a-c can beprogrammed to automatically execute tasks such as emitting light and/orcollecting image data of the subject or combinations thereof atpredetermined time points upon reaching their designated positionswithin the pattern. In an embodiment, device 100 can activate only someof the pucks 104 a-c to emit light and/or collect image data at specifictime intervals. In an embodiment, a combination of the aforementionedevents can occur wherein the pucks 104 a-c are pre-programmed to executethe tasks related to collecting image data at particular time pointswhen the device 100 can interfere with their functioning to deactivateor activate an otherwise idle drone to emit light and/or collect imagedata based for example, on user input. Therefore, the selected pucks anddrones are activated and the image data is captured.

In some embodiments, a three dimensional model of a selected pattern canbe generated/simulated by the device 100 based on user input. In anembodiment, the selected pattern can be simulated by the processor withreference to the location of the subject to be imaged. The positions forthe drones 902 a-n within a selected pattern can either be determined bythe device 100 alone or in combination with a human operator in anembodiment. In one embodiment the user can determine where particulardrones 902 a-n should be placed within the pattern, for example, byclicking at the particular points on a 3D model displayed on a puckdisplay screen. The processor can be configured to store the coordinatesof the points receiving the user clicks or touches. The positions in thepattern may be defined in terms of various coordinate systems, e.g.,Cartesian coordinates or spherical coordinates. In differentembodiments, the puck can be configured to suggest drone patterns foruser selection or even automatically select certain imaging patternsbased on the attributes of the subject such as but not limited to theshape of the subject, nature and the area of surface being imaged. In anembodiment, certain imaging requirements such as the type of puck to bepositioned can be associated with the selected positions. For example, adefault imaging requirement of having a light source can be associatedwith each of the selected positions within the pattern. Such imagingrequirements of the positions can be further modified based on userinput. Thus, a user can specify if a light source, a camera or theircombination with a filter should be placed at each position and anyparticular settings to be associated with such equipment. Upon receivingthe imaging requirements for the positions, the identification andattribute data of the drones and/or pucks can be selected by a user forpositioning is received. By way of illustration and not limitation, theuser can select the drones having appropriate pucks for the formation ofthe pattern to collect image data. In an embodiment, each drone and/orpuck can be uniquely identified via a respective ID which can alsoindicate its attributes such as the imaging components it has on thepuck. The identification data from the drones can be obtained viacommunication technologies such as but not limited to, Bluetooth orWi-Fi. For example, the drones or pucks can have their identificationand attribute information encoded on respective passive or active RFID(radio frequency identification) tags in order to provide the positionand attribute data. The device 100 can determine if the selecteddrones/pucks and their attributes match the previously received imagingrequirements for the positions. For example, if the selected pattern andposition requirements include ten drones, two with camera-equipped pucksand eight with light source-equipped pucks, device can determine ifthere are ten drones that satisfy the specified requirements. In case itis can be determined that the drones selected for pattern formation donot match the specified requirements, a user notification can begenerated and the user can be provided an opportunity to rectify theerror. If it is determined that the selected drones satisfy therequirements, the position data and imaging requirements are transmittedto the drones. In one embodiment, each drone can receive only dataassociated with its position and its respective puck imaging settings.In an embodiment, the entire position and imaging data set istransmitted to all the drones/pucks which can recognize or obtain theirrespective data from the received data set. When the process ofobtaining and transmitting the position and imaging data is complete andthe subject to be imaged is appropriately positioned, the drones can beactivated for positioning or for pattern formation.

When collecting image information, position information or data such ascoordinates of a position in a particular pattern and imagingrequirements associated with the position are received by a drone. In anembodiment, the position coordinates of the drone can be defined withrespect to one or more reference entities based on different factorssuch as but not limited to, the pattern to be formed or the subject tobe imaged, the location at which the subject is being imaged orcombinations thereof. In addition, the imaging requirements such as, thesettings of the puck including but not limited to, brightness of thelight sources, angle and focus of the light sources or cameras, can alsobe received. An activation signal to form the pattern can be receivedfrom device 100. Reference entities with respect to which thecoordinates are defined and the pattern is to be formed are identified.In an embodiment, only a single reference plane such as the ground maybe sufficient to form the pattern. However, a drone can require morethan one reference entity to identify its position. For example, one ormany radio sources giving out radio emissions can be placed at thelocation of the subject so that the drone can employ the ground and theradio source(s) as references to identify its destination point in thethree dimensional space. The drone maps the trajectory to itsdestination position. A computational module can be included in theprocessor which can receive or identify the position data, referencelocation data and the current location of the drone as input and map atrajectory from the current location to the destination. Variousalgorithms now known or to become known can be employed by the dronesfor independent trajectory planning and tracking. In accordance with onealgorithm, to generate dynamically feasible trajectories, an initialplan is generated through the environment which satisfies collision andobstacle avoidance constraints. Such algorithm can further allow forreal-time planning in cluttered environments also based on techniquessuch as visibility graphs. The resulting trajectories are defined insimple geometric terms of lines and connecting curves with accompanyingdesired velocities along each segment. Then, a feasible set of inputsand travel speeds is computed based on the curvature of the path, givenspeed and acceleration constraints on the vehicles. A drone can thennavigate to and reach its destination. In an embodiment, the progress ofa plurality of drones is monitored by device 100 as they navigate totheir destinations to provide feedback in case of a deviation or animpending collision. Upon reaching the destination, the drone canlevitate at the destination position and await a signal that indicatesthe commencement of the imaging procedure. In an embodiment, theprogress of the drones can be monitored and upon the all the dronesreaching their respective destinations and forming the complete pattern,a signal to begin the imaging procedure can be received by a drone asshown at 612. In an embodiment, the instruction set for the entireimaging procedure can be provided to each of the drones and a drone canidentify its particular instructions from the received instruction set.Such identification can either occur due to the drone identifier beingassociated with the instructions or due to the position informationassociated with the instructions. Accordingly, puck carried by a droneis activated in accordance with the received instructions in order toexecute tasks such as illuminating the subject or collecting the imagedata or combinations thereof.

In general, upon reaching their destinations the drones 902 a-n levitateor hover or float in the air over the subject in their respectivepositions as they execute instructions to collect image data of asubject. In an embodiment, the a plurality of drones 902 a-n are usedonly for illuminating the subject while the image data is collected byother drones 902 a-n. Various kinds of image data such as a still imageor a video can be collected in accordance with the embodiments disclosedherein.

In some embodiments, drones 902 a-n can orbit around an object to beimaged to create a virtual sphere of drones 902 a-n executing one ormore of the tasks including illuminating the object or collecting imagedata as they move along their respective trajectories. One drone canhover above the object to be imaged in its position to collect or aidthe collection of image data. Thus, a pattern of drones 902 a-n can alsobe formed wherein some of the drones 902 a-n move in particulartrajectories around the subject to be imaged and some of the drones 902a-n simply levitate or hover above the subject while image data is beingcollected.

In some embodiments, drones 902 a-n can be arranged in a grid patternformed by a plurality of the drones 902 a-n in accordance with oneembodiment. The subject to be imaged (not shown) can be situated infront of the pattern and may be imaged via pucks. The planar gridformation can then be moved so that the plane of the grid formation maybe oriented at myriad positions relative to a horizontal or verticalreference plane.

In some embodiments, drones 902 a-n can be arranged in a sphericalarrangement of the drones 902 a-n in accordance with one embodiment. Thesubject being imaged can be situated inside the sphere in oneembodiment. Thus, the subject can be initially positioned and theplurality of drones arrange themselves in a plurality of substantiallyevenly distributed circles to form the spherical pattern around thesubject employing one or more reference entities as detailed herein.

In some embodiments, it can be further appreciated that drones 902 a-nin a given pattern need not be identical and that different drones ofdifferent sizes and various attributes (such as by way of non-limitingexample, weight carrying capacity, flight duration capability, inertialcharacteristics, processor capacity, among other characteristics) can beused at different positions in a single formation.

FIGS. 10A-B illustrates a puck with a display configured to act as aholographic, light field display.

As illustrated in FIG. 10, a display 114 (as illustrated in FIGS. 3Dthrough 3R), is connected to puck 104 via a first rotatable connectorsuch as a ball and socket connection. The first rotatable connector isconnected to a telescopic member. In some embodiments, telescopic memberis connected to display 114 via a second rotatable connector. Asdiscussed previously, display 114 can comprise an OLED display devicewhich can be single sided, transparent, double sided, or have a displayside and a mirror side.

In one embodiment, the telescoping member allows for the raising andlowering of display 114 away from puck 104. In some embodiments, thetelescoping member can be controlled manually and/or programmatically bypuck 104. In some embodiments, the telescoping member can be tilted ateither the first rotatable connector or second rotatable connector, orboth. Alternatively, or in conjunction with the foregoing, thetelescoping member can be flexible. In some embodiments, the rotatableconnectors can include one or more copper slip rings to allow for datacommunications between puck 104 and display 114 or another componentthat the telescoping member might be connected to.

In some embodiments, display 114 can be rotated via a motor present ator near the point of connection between the display 114 and puck 104(e.g., proximate the rotatable connector). In other embodiments themotor can be at a distal or intermediate position relative to the OLEDand the OLED rotatably driven by a flexible drive shaft that extendsthrough the telescoping or flexible member.

In some embodiments, the motor is configured to spin at up to 18,000 RPM(300 RPS), although the precise number of revolutions is not intended tobe limiting. In one embodiment, the speed of the motor is controlled bypuck 104 by varying the voltage supplied to the motor. Alternatively, orin conjunction with the foregoing, display 114 can be spun whileparallel with the surface of puck 104.

In some embodiments, transparent members or a transparent cage cansurround the display 114 to prevent injury to users. In someembodiments, the goggle portion (illustrated and discussed previously)can be utilized to protect a user from the spinning display. In someembodiments, a device 100 equipped with two attached pucks can provide abinocular stereo display when both pucks are providing the spinningdisplay (e.g., both displays are spinning in front of the user's eyes)

While spinning, display 114 can be controlled by one or more processingdevices within puck 104 (e.g., a Qualcomm 835 SNAPDRAGON® or NVIDIATEGRA® X1 processor). Puck 104 (via one or more processing elements) cantransmit image data to display 114 and cause display 114 to generate alight field display due to the spinning of the display 114. In someembodiments, puck 104 transfers image data to display 114 at 300 framesper second, although the specific frame rate is not intended to belimiting. In one embodiment, puck 104 is capable controlling rasterlines on display 114 in order to generate the light field display.Alternatively, or in conjunction with the foregoing, puck 104 canrapidly turn the display 114 on and off, thus providing a holographiclight field display via the spinning OLED display 114.

In some embodiments, the puck 104 can change the contents of the display114 while spinning based on the position of a user. As discussed above,when the display 114 is rotating in front of a user's eye (e.g., whenthe puck 104 is connected to the device 100), the display 114 canprovide a light field display before a user while the user is in motion.Additionally, in this embodiment, the puck 104 can be configured totrack a user's eye movements and adjust the position and/or RPMs of thedisplay 114 as necessary. Additionally, the puck 104 can also track theuser's position and update the content of the spinning display 114 basedon the position of the user.

Although described in the context of a spinning OLED display, otherembodiments exist that can be utilized to provide a similar effect. Inone embodiment, display 114 can include a mirror on the underside ofdisplay 114. In alternative embodiments, both sides of display 114 canbe mirrors. In this embodiment, the puck 104 can utilize one or moreprojectors to project an image onto the mirror(s) to produce a similarholographic display. In some embodiments, the display 114 can, itself,be utilized as a mirror. In some embodiments, the display 114 can beutilized as a mirror on the OLED portion of the display 114 while amirror section is present on the opposite side of the display 114. Insome embodiments, the display 114 can be double sided wherein both sidesare utilized as displays or as mirrors.

FIGS. 11A-C illustrate a puck with a display configured to display abody- or object-based virtual display, according to some embodiments ofthe disclosure.

As discussed previously, in some embodiments, pucks 104 a and 104 b areequipped with one or more depth sensing camera devices. In theembodiment illustrated in FIG. 11A, the one or more depth sensing cameradevices are configured to track the position of a body part of the user,such as a hand, thigh or forearm 1104. As discussed previously, thedepth sensing cameras allow the puck 104 a to identify the preciselocation of a user's body part (e.g., forearm 1104) or other object withrespect to the puck 104 a and/or one or more projection devices(discussed previously) present on puck 104 a. Although the followingdisclosure discusses a user's forearm 1104, other body parts can beutilized as a display surface. For example, the fingers of a user onforearm 1104 can be used as a display surface alternatively, or inconjunction with, the forearm 1104 of a user.

In the illustrated embodiment, the puck 104 a displays a rectilinear orconical or other beam shaped projection 1102 into a space in front ofthe user and puck 104 a. As discussed previously, the puck 104 a isequipped with one or more projection devices to provide such aprojection. Although illustrated as rectilinear, other shapedprojections can be projected by puck 104 a (e.g., a circularprojection). Notably, however, projection area 1102 is larger than auser's forearm 1104 and thus allows for a user's arm to movesubstantially freely while remaining within projection 1102.

In some embodiments, a single projector provides projection 1102.Alternatively, the puck 104 a can utilize multiple projectors to provideprojection 1102. In one embodiment, projection 1102 can comprise a1280×1440 pixel projection, although the specific size of projection1102 is not intended to be limiting. In some embodiments, projectors onpuck 104 a are stationary while in other embodiments the projectors canmove. In some embodiments, projectors on both pucks 104 a, 104 b can beutilized to increase the size and/or overlap of projection 1102.

As illustrated, the output of the projection devices of puck 104 a isconfigured to display an interactive interface area 1106 on a user'sforearm 1104. In some embodiments, the interface area 1106 displays thedisplay of a computing device such as a laptop, mobile phone, tablet, orother device, or the input device for a computer or smart device, suchas a touch pad area or projected keyboard or buttons or icons, or aremote control for a puck in set top box mode, or a video gamecontroller. In this embodiment, a computing device is connected to puck104 a via a wireless connection such as a BlueTooth, Wi-Fi, or otherconnection. The computing device can be configured to transmit itsdisplay to puck 104 a via the wireless connection. For example, a mobilephone can be configured to “mirror” its display to puck 104 a. Inresponse, puck 104 a can include the mirrored display in projection1102. In some embodiments, puck 104 a is configured to modify the color,hue, or saturation of a projection 1102 based on the skin color of theuser. In some embodiments, projection 1102 can be toggled based ondetecting the presence of a body part (such as the user's arm).

In one embodiment, the output of puck 104 a (via one or more projectors)can be manipulated such that the desired interface area 1106 onlyoccupies a portion of the viewable projection 1102. In this embodiment,puck 104 a receives the location of the user's forearm 1104 (via depthsensing cameras or other motion tracking or sensing elements, e.g.visible or invisible markers) and translates the position of the user'sforearm 1104 into a set of coordinates located within projection 1102.These coordinates can be of various forms, but generally define ageometric shape such as a polygon representing a portion of the user'sforearm 1104 as a planar portion of projection 1102. After identifyingthe portion of projection 1102, puck 104 a transforms the desireddisplay to fit within the interface area 1106. For example, the puck 104a can scale a mobile device screen to match the dimensions of the areaof interface 1106 and a user can dial a phone or call up an app byinteracting with the forearm projection.

Although described in the context of mirroring a device, interface area1106 can display any other type of digital content, including contentnot transmitted by a user's computing device. For example, the puck 104a can be configured to display a movie or television show in interfacearea 1106. In this embodiment, the puck 104 a can receive the movie ortelevision show or video game from a remote source (e.g., a web site orweb service) via a Wi-Fi interface on the puck 104 a and transform thevideo source to fit within interface area 1106.

In some embodiments, projection 1102 is “clipped” to a user's forearm1104. As discussed previously, the puck 104 a is configured to track theposition of a user's forearm 1104. Additionally, puck 104 a can beconfigured to detect the rotation of a user's forearm 1104. Upondetecting a rotation, puck 104 a transforms the display such that theinterface area 1106 continues to display the same projection regardlessof the movement of a user's forearm. Alternatively, puck 104 a can beconfigured to update the contents of a display in unison with the user'sforearm movement and thus simulate a display that “wraps around” theuser's forearm. In some embodiments arm movements can be controlsignals, such as for example a user rotating one's arm can cause thedisplay to scroll. In some embodiments, the puck 104 a can be configuredto “pin” interface area 1106 to a particular point on the user's body(e.g., at the user's wrist).

In some embodiments, interface area 1106 is utilized as an input device.As discussed previously, the puck 104 a is configured to monitor theposition of various body parts of the user. As part of this monitoring,in some embodiments, the puck 104 a can be configured to track theposition of a user's hand and/or fingers. In this embodiment, the puck104 a monitors the position of a user's fingers 1108 and can detect thatone or more of the user's fingers 1108 are interacting with a displayprojected on interface area 1106. In response, the puck 104 a can detectwhere on interface area 1106 a user has touch and can generate asimulated input signal. For example, if interface area 1106 isdisplaying a “home” screen of a mobile device, puck 104 a can detectthat one of user's fingers 1108 is attempting to “touch” an icon basedon the position of the user's finger. In this example, puck 104 atransforms positional coordinates of the user's finger to a Cartesiancoordinate within the display projected in interface area 1106. Next,puck 104 a transmits this coordinate to a user's computing device toinstruct the computing device to “touch” at the calculated coordinate,thus simulating a “touch” even on the mobile device. Other deviceexamples are television remotes, video game controllers, calculators,tablets, e-readers and the like.

FIGS. 11B and 11C illustrate a puck configured to operate in an “x-ray”mode of operation according to some embodiments of the disclosure.

As illustrated in FIG. 11B, puck 104 a displays projection 1102 whichincludes an interface area 1110 a projected onto a user's forearm 1104.As discussed previously, and as illustrated in FIG. 11B, interface area1110 a comprises an interactive simulated touchscreen display(represented by numerals 1, 2, 3 4). For example, each number ininterface area 1110 a can comprise an input element (e.g., an icon orsimilar element). In the embodiment illustrated in FIG. 11B, interfacearea 1110 a is projected onto user's forearm 1104. Thus, to the user,interface area 1110 a appears to be displayed on the user's forearm1104, while the forearm is still visible to the user.

FIG. 11C illustrates a projection operating in a translucent ortransparent mode according to some embodiments.

In the embodiment illustrated in FIG. 11C, the projection 1102 can begenerated based on images recorded by puck 104 b. Specifically, in theillustrated embodiment, puck 104 b (and/or 104A) is configured tocapture live images of table 1112 and transmit these images to puck 104a. In response, puck 104 a generates a projection 1102 based on thecaptured images. In the illustrated embodiment, projection 1102comprises a rectilinear projection that corresponds to the scene infront of the user from the user's perspective. As discussed above, pucks104 a, 104 b are configured to track the user's position and eyemovements and thus can update the recorded images to match the scene infront of the user (e.g., by adjusting the angle of cameras on pucks 104a, 104 b).

As illustrated in FIG. 11C, when a user's forearm 1104 is moved into theprojection 1102, the user's forearm 1104 is projected upon by puck 104 awith the images recorded by puck 104 b. Thus, in one embodiment, theprojection on the user's forearm 1104 corresponds to the scene in frontof the user's forearm 1104 (e.g., table 1112). Thus, from theperspective of the user, the user's forearm 1104 appears transparent(e.g., in “x-ray” mode) or translucent since the portion of the tableotherwise obscured by the user's forearm is “seen around” by the camerasand projected on the forearm to give the impression that the forearm isnot there.

In some embodiments, the projection 1102 results in the users forearm1104 being “translucent” from the perspective of the user. Specifically,as illustrated in FIG. 11C, interface area 1110 b includes the samecontents of the displayed interface area 1110 a. Notably, however,interface area 1110 b is translucent, allowing for the display of table1112 instead of the user's forearm 1104. As discussed previously, puck104 b is configured to capture, for example, images of table 1112. Thus,when operating in the mode illustrated in FIG. 11C, the projection 1102can “erase” the user's forearm 1104 by projecting the images of thetable 1112 upon the user's forearm 1104.

FIGS. 12A-H illustrate a puck with a wand-mounted display configured toact as a holographic, light field display, according to some embodimentsof the disclosure. As illustrated in FIGS. 12A-F, puck 104 is configuredto be used in a “tabletop” mode wherein legs 208 a, 208 b, and 208 c areplaced on a flat surface such as a table or the palm of a user's hand.

As discussed previously in connection with FIGS. 10A-B, a puck 104includes a wand 1208 connected to puck 104. In some embodiments, wand1208 is bendable and/or rotatable. In some embodiments, wand 1208 can betelescopic (as discussed previously in connection with FIGS. 10A-B). Insome embodiments, wand 1208 is detachable from puck 104 as illustratedin FIG. 12G. In some embodiments, wand 1208 is detachable from puck 104via a physical connection and/or electrical connection. In oneembodiment, an electrical connection between wand 1208 and puck 104comprises a USB-C® interface. In some embodiments, wand 1208 includes aninternal drive shaft that is connected to a motor in puck 104 (via oneor more interlocking gears) and to connection point 1204 via casing1206. In some embodiments, casing 1206 houses additional electricalcomponents as discussed more fully in connection with FIG. 12G.

In one embodiment, one or more slip rings or slip ring platterselectrically couple a rotating output portion of the motor drive in puck104 to an output of one or more processing elements in puck 104.Likewise, a second set of slip rings connect the rotating end in casing1206 to an electrical output connected to connection point 1204. Thus,one or more processing devices in puck 104 can transmit data signals todisplay 1202 while the drive shaft in wand 1208 is in motion, which inturn rotates OLED 1202. In alternative embodiments, the drive shaft inwand 1208 can include one or more copper collars that isolate electricaldata paths from the mechanically moving portions of the drive shaft.

In some embodiments, wand 1208 is additionally configured to beconnected to a VR/AR device (as discussed previously) without utilizinga puck 104. In this embodiment, wand 1208 is connected to the VR/ARdevice via a USB-C® interface or slip rings as discussed. Specifically,in some embodiments, the wand 1208 can be connected to the VR/AR devicein place of goggle portion 110. Notably, as discussed previously, insome embodiments, goggle portion 110 is connected to the VR/AR devicevia a USB-C® interface which allows for the transfer of data (e.g.,three-dimensional scenes) from a puck (or headband portion) to goggleportion 110. In one embodiment, the goggle portion 110 is removed fromear piece portion 120 and display 1202 (via wand 1208) is connected toear piece portion 120 in place of goggle portion 110. In thisembodiment, the VR/AR device transmits image data to display 1202 (viawand 1208) using a USB-C® interface. Thus, in some embodiments, a VR/ARdevice (as depicted previously) does not include a puck 104 or a goggleportion 110 and drives display 1202 via wand 1208 without furthercomponents.

As illustrated, wand 1208 is connected to display 1202 via casing 1206and connection point 1204. In some embodiments, connection point 1204comprises a ball and socket connector. In this embodiment, connectionpoint 1204 allows display 1202 to be tilted or otherwise moved inrelation to casing 1206 and wand 1208. In some embodiments, connectionpoint 1204 can be configured to “snap” into one or more positions. Forexample, when snapped into a 0 degree position, display 1202 will spinas depicted in FIG. 12B. Alternatively, when snapped into a 90 degreeposition, display 1202 will spin as depicted in FIG. 12D. Alternatively,when snapped into a 45 degree position, display 1202 will spin asdepicted in FIG. 12F. Alternatively, or in conjunction with theforegoing, casing 1206 can be connected to wand 1208 via a similarconnection (e.g., a ball and socket connection). In the illustratedembodiment, connection point 1204 is connected to the underside ofdisplay 1202. In some embodiments, casing 1206 includes one or more sliprings, slip ring platters, or copper collars as discussed previously toallow for data communications between display 1202 and puck 104.Alternatively elements 1202, 1204 and 1206 can be removably mountable asan assembly to wand 1208 and different angled OLED assemblies can beplaced on wand 1208 as desired to achieve the different rotation anglesdepicted in FIGS. 12B, 12D and 12F. Each angled assembly will connect tothe drive shaft and be electrically connected to conductors in the wandvia slip rings or other rotating electrical connections as describedherein or otherwise known in the art.

As discussed previously, a motor in puck 104 can be configured to spinat up to 18,000 RPM (300 RPS), although the precise number ofrevolutions is not intended to be limiting. In one embodiment, the speedof the motor is controlled by puck 104 by varying the voltage suppliedto the motor. As discussed previously, the extension, rotation, and/ortilt of wand 1208 and/or display 1202 can be controlled manually by auser or programmatically by puck 104. Details of the motor in puck 104are described more fully in connection with FIG. 12H.

As illustrated more fully in FIGS. 12A and B, display 1202 rotates viaconnection point 1204. In the illustrated embodiment, display 1202rotates around an axis orthogonal to the display 1202. That is, asillustrated, display 1202 rotates about an axis parallel to the surfaceof puck 104, orthogonal to display 1202, and extending out fromconnection point 1204. In alternative embodiments, display 1202 can betilted such that display 1202 rotates about an axis orthogonal to thesurface of puck 104 and of the display 1202. That is, in thisembodiment, display 1202 is positioned parallel to the upper surface ofpuck 104. In alternative embodiments, since display 1202 is connected towand 1208 via connector 1204 (e.g., a ball and socket connector),display 1202 can be tilted or otherwise moved such that display 1202rotates about axes at varying degrees with respect to the surface ofpuck 104 as illustrated in FIG. 12F.

In some embodiments, display 1202 has a resolution of 720×720 pixels,thus when spinning rapidly, a spinning display doubles the resolutionusing a single display 1202. In this embodiment, the puck 104 controlsthe output on display 1202 based on an identified position of thedisplay 1202. As described in more detail herein, a grey encoder can beconnected to the motor in puck 104 to identify the position of thedisplay 1202. Given that the display 1202 is rotating rapidly, theoutput of display 1202 can be timed such that a single image isdisplayed visually to a user wherein the single image occupies acircular area having a diameter twice the diameter of display 1202.

FIGS. 12C-D illustrate a puck with a display configured to act as aholographic, light field display, according to some embodiments of thedisclosure.

As illustrated, a puck 104 is equipped with a wand 1208, casing 1206,connection point 1204, and display 1202. Details of wand 1208, casing1206, connection point 1204, and display 1202 were discussed previouslyin connection with FIGS. 12A-B and are incorporated herein by reference.

FIGS. 12C-D illustrate an alternative connection point 1204 between wand1208 and display 1202. In the illustrated embodiment, display 1202comprises a two-sided OLED display. In alternative embodiments, display1202 comprises a single-sided OLED display.

Display 1202 is connected to casing 1206 via connection point 1204. Inthe illustrated embodiment, connection point 1204 is connected at theedge of display 1202. In alternative embodiments, connection point 1204includes an arm attached to one side of display 1202 and extending froma point on the edge of display 1202 toward the center of display 1202along a radius of display 1202. In this embodiment, connection point1204 allows for the rotation of display about an axis extendingoutwardly from connection point 1204, extending across the diameter ofdisplay 1202, and orthogonal to the surface of puck 104.

In the embodiments illustrated in FIGS. 12A-F, while spinning, display1202 can be controlled by one or more processing devices within puck 104(e.g., a Qualcomm 835 SNAPDRAGON® or NVIDIA TEGRA® X1 processor). Puck104 (via one or more processing elements) can transmit image data todisplay 1202 and cause display 1202 to generate a light field displaydue to the spinning of the display 1202. In some embodiments, puck 104transfers image data to display 1202 at 300 frames per second, althoughthe specific frame rate is not intended to be limiting. In oneembodiment, puck 104 is capable of controlling raster lines on display1202 in order to generate the light field display. Alternatively, or inconjunction with the foregoing, puck 104 can rapidly turn the display1202 on and off, thus providing a holographic light field display viathe spinning OLED display 1202.

In some embodiments, the puck 104 can change the contents of the display1202 while spinning based on the position of a user. As discussed above,when the display 124 is rotating or spinning, the display 1202 canprovide a light field display before a user while placed upon a flatsurface. Additionally, in this embodiment, the puck 104 can beconfigured to track a user's eye movements and adjust the positionand/or RPMs of the display 1202 as necessary. Additionally, the puck 104can also track the user's position and update the content of thespinning display 1202 based on the position of the user.

FIGS. 12E and 12F illustrate alternative embodiments of a puck with awand-mounted display configured to act as a holographic, light fielddisplay, according to some embodiments of the disclosure.

As illustrated in FIG. 12E, display 1202 is connected to puck 104 viaconnection point 1204, casing 1206, and wand 1208 as discussedpreviously. In the illustrated embodiment, connection point 1204 cancomprise a ball and socket connection point. In alternative embodiments,connection point 1204 can additionally include a hinged arm extendinginward toward the center of display 1202 along a radius of display 1202.In this embodiment, the connection point 1204 can be configured to actas a hinge allowing the display 1202 to rotate in a circular motion asdepicted in FIG. 12B as well as in a hinging motion depicted in FIG.12F.

As illustrated in FIG. 12F, connection point 1204 allows for themovement of display 1202 inward and outward from an axis orthogonal tocasing 1206. In the embodiment illustrated in FIG. 12F, display 1202 canbe configured to move about both axes simultaneously, resulting in afunnel or bowl-shaped movement (appearing like a parabolic radar dishshape for example).

In some embodiments, wand 1208 includes various other components such asLED markers, OLED displays, QR codes, or reflective portions. Asillustrated in FIG. 12G, wand 1208 can include LED or OLED markers 1210at various positions of the wand 1208 including within the casing 1206and at various points along the longitudinal axis of the wand 1208. Inthese embodiments, the various other components allow for puck 104 totrack the wand 1208 (via camera devices on puck 104, as discussedpreviously). Specifically, puck 104 tracks the position and movements ofwand 1208 via these various other components (e.g., LED markers). Inthese embodiments, wand 1208 can be utilized by a user as a laserpointer, “magic wand,” drumstick, “lightsaber,” or other handheld inputmechanism. In some embodiments, when puck 104 tracks the position ofwand 1208, the position of wand 1208 can be utilized to update athree-dimensional display. In some embodiments, the position of wand1208 can be utilized to generate a digital representation of wand 1208in a three-dimensional scene. For example, a three-dimensional game(e.g., a “wizard”-oriented game) can be displayed by the VR/AR device(e.g., via goggle portion 110 or a display 1202) while the wand 1208 canbe tracked and utilized as a “magic wand” which can be displayed beforethe user in the game as an controllable input (e.g., in a first-personshooter mode). In some embodiments, wand 1208 can be given to otherusers and tracked in similar manners. In some embodiments, since a VR/ARdevice is equipped with pucks (and thus, two wands), one user canutilize a first wand while another utilizes a second wand. In thisembodiment, a wand 1208 manipulated by a second user can be tracked andrepresented in a three-dimensional display.

FIG. 12H is a system diagram illustrating a system for controlling amovable wand according to some embodiments of the disclosure.

As illustrated in FIG. 12H, SoC 1212 is connected to motor controller1214. In some embodiments, SoC 1212 can comprise a Qualcomm 835SNAPDRAGON®, or the NVIDIA TEGRA® X1 processor as discussed previously.In embodiments, SoC 1212 is located within a puck as discussedpreviously.

As described above, SoC 1212 is communicatively coupled to motorcontroller 1214. In one embodiment, motor controller 1214 can comprise amicrocontroller, microprocessor, programmable logic controller or othersuitable electronic device configured to operate motor 1216. In theillustrated embodiment, SoC 1212 transmits data to motor controller 1214to control the speed (e.g., RPMs) of the motor 1216 via motor controller1214. In one embodiment, motor 1216 may comprise any suitable electricmotor.

A grey code wheel 1218 is connected to one or more of the gears on motor1216. Use of a grey code wheel 1218 allows the system to monitor theprecise position (e.g., rotational position) of the external gear ofmotor 1216. Encoder 1220 is configured to read the grey code wheel 1218and generate a digital representation of the position of the gear ofmotor 1216. Encoder 1220 transmits this position to motor controller1214 which transmits the positional information to SoC 1212.

As illustrated above, the SoC 1212 controls the speed of the motor 1216and is informed by motor controller 1214 (via encoder 1220) of theprecise rotational position of the motor 1216. As discussed previously,a gear of motor 1216 is connected to an OLED display via a driveshaft orsimilar apparatus. Thus, SoC 1212 (through encoder 1220) is constantlyupdated with the precise angle of the OLED display 1222, enabling theSoC or other controller to send video data to the rotating OLEDappropriate to its location to create holographic or other light fieldor video display effects.

SoC 1212 is additionally communicatively coupled to OLED display 1222via a MIPI, SIPI or similar connection. In the illustrated embodiment,SoC 1212 drives the display on OLED display 1222 via this connection.Since SoC 1212 know the precise position and angle of OLED display 1222,SoC 1212 updates the display of OLED display 1222 based on this positioninformation in order to generate a holographic display as discussedpreviously. Additionally, SoC 1212 receives other inputs that may beused to adjust the display of OLED display 1222 such as the position ofa user, the eye position of a user, ambient lighting conditions, andother inputs as discussed previously.

FIGS. 13A-F illustrate a puck with a display configured to act as avapor-based holographic, light field display, according to someembodiments of the disclosure.

FIG. 13A illustrates a vapor projection component 1302 nested in acavity of puck 104. In this embodiment, vapor projection component 1302can be detached and attached to cavity of puck 104. In one embodiment,vapor projection component 1302 connects to puck 104 via a USB-C® orother wired connection on the bottom of vapor projection component 1302and on the outer surface of the inner cavity of puck 104.

As illustrated in FIG. 13A, vapor projection component 1302 includes asurface 1304 capable of emitting one or more jets of vaporized water orheated air. Operation of the vapor projection component 1302 via surface1304 is described more fully in connection with FIGS. 14A-E.

FIG. 13B illustrates an alternative configuration of a vapor-basedholographic, light field display. In the illustrated embodiment, vaporprojection component 1302 is inserted into a cavity of puck 104. Duringoperation, one or more jets of vaporized water or heated air areexpelled outwardly from the surface 1304 of vapor projection component1302 and into the air above the puck 104. In some embodiments, asdiscussed more fully herein, the jets of vaporized water (or heated air)may form various shapes such as a line, cylinder, column or other solid.

During the operation of the vapor jets, the puck 104 displays images viaprojectors 320 a-c. By projecting appropriately controlled light beamsonto a stream of vapor jets, puck 104 can display a holographicprojection into the air above it containing the vaporized water asillustrated in more detail in FIGS. 13D-F. In some embodiments,projectors 320 a-c are movable and may adjust their angle based ontracking the user, the vapor pattern or other needs. In one embodiment,projectors 320 a-c are under the control of one or more processingelements in puck 104. In this embodiment, puck 104 may coordinate thedisplay projected by projectors 320 a-c with the active vapor jets asdiscussed more fully in connection with FIGS. 14A-E.

FIG. 3C illustrates the insertion of vapor projection component 1302according to some embodiments of the disclosure. In the illustratedembodiment, vapor projection component 1302 can be removed from puck 104resulting in a cavity 1306 being present in puck 104. In someembodiments, cavity 1306 can be configured as a universal receptacle(e.g., via a USB-C® interface) to receive vapor projection component1302 and other components fitted to the dimensions of the cavity 1306.In some embodiments, vapor projection component 1302 can be configuredto rotate as needed when placed in cavity 1306. In this embodiment,processing elements in puck 104 coordinate the display from projectors320 a-c to enable a rotating display on the vapor jets expelled fromsurface 1304.

FIGS. 13D-F illustrate various vapor-based projections from a puckaccording to some embodiments of the disclosure.

FIGS. 13D and 13E illustrate a cylindrical or columnar projection ofvapor jets 1312 from surface 1304. In some embodiments, surface 1304 mayinclude multiple apertures for allowing streams of vapor to be emittedfrom vapor projection component 1302. In some embodiments, the streamsof vapor may be emitted in a timed fashion. As illustrated, a projection1314 can be displayed by one or more projectors present on puck 104. Inone embodiment, streams of vapor may be staggered rapidly to simulate asingle projection of vapor. That is, certain subsets of the apertures insurface 1304 may be activated via a mask or screen in a timed andcoordinate fashion. In this manner, the projection may likewise be timedsuch that the projection is coordinate with the apertures beingactivated at any given moment.

FIG. 13F illustrates an alternative vapor-based projection from a puckaccording to some embodiments of the disclosure.

As illustrated in FIG. 13F, a vapor projection component 1302 isconfigured to expel vapor from surface 1304 in a single line of jetsresulting in a planar jet “sheet.” 1312. In one embodiment, surface 1304contains a matrix of apertures while a movable belt drags a mask acrossthe surface 1304 enabling only a single row of apertures at a time.Operations of a belt-based jet system are described more fully inconnection with FIGS. 14A-E. As illustrated, projectors 320 a-c areconfigured to project one or more images on a sheet 1312 of vapor jets.In one embodiment, the images projected by projectors 320 a-c can beappropriately timed based on which row of surface 1304 is activated.

FIGS. 14A-E illustrate a vapor-based projection puck according to someembodiments of the disclosure.

FIG. 14A illustrates a fixed planar vapor-based projection puckaccording to some embodiments of the disclosure.

As illustrated in FIG. 14A, a vapor projection module 1402 is configuredto project vapor (or steam) upward to allow for the projection of animage. As discussed previously, vapor projection module 1402 is centeredwithin a cavity of a puck and can be inserted and removed as needed.

In the illustrated embodiment, a micro dehumidifier 1404 is used toextract water vapor from the air. In some embodiments, microdehumidifier 1404 comprises a solid-state micro dehumidifier. In oneembodiment, micro dehumidifier 1404 is placed within a puck. Inalternative embodiments, micro dehumidifier 1404 is placed within or onother components of a VR/AR device such as the headband. In someembodiments, micro dehumidifier 1404 is capable of being controller by aprocessor (or other device) within a puck or within a VR/AR device. Forexample, micro dehumidifier 1404 can be turn on or off based on thewater level of water tank 1406. In alternative embodiments, microdehumidifier 1404 can be coupled to a reservoir connected to pipe 1408and situated between fan 1410 and dehumidifier 1404. In this embodiment,the reservoir can store water and be controlled by a processing elementto refill tank 1406 as needed. In some embodiments, the reservoir islocated in other portions of a VR/AR device such as headband portiondiscussed previously.

Micro dehumidifier 1404 is connected to water tank 1406 via water pipeor tube 1408. In one embodiment, water pipe 1408 comprises a flexibletube that is capable of routing water from micro dehumidifier 1404 towater tank 1406. In some embodiments, water pipe 1408 can be routed froma headband portion of a VR/AR device to water tank 1406. In theillustrated embodiment, water tank 1406 is located within a puck device.As discussed previously, water tank 1406 can include one or more sensorsthat monitor the water level in the tank 1406. In some embodiments, aprocessing device within the puck can monitor the water level of tank1406 and/or can control the operation of micro dehumidifier 1404 basedon the detected water level.

Piezo transducer 1412 atomizes the water and rapidly converts the liquidwater into liquid vapor or steam. In some embodiments, transducer 1412can be submerged and fixedly connected to the edges of tank 1406. Insome embodiments, transducer 1412 may be fixed to the bottom of tank1406. In some embodiments, transducer 1412 is not connected to tank 1406and rests within tank 1406. As illustrated, vapor projection module 1402may be partially covered with the exception of a single row of laminarflow tubes 1414.

Fan 1410 is located beneath water tank 1406. In the illustratedembodiment, fan 1410 comprises a speed-controlled fan. In someembodiments, fan 1410 is controlled by a processor in a puck or in othercomponents of the VR/AR device. In some embodiments, processor monitorsthe operation of fan and utilizes the operational status of the fan tocontrol the projection from projector 1416. For example, a processingdevice in a puck can toggle projections on or off based on detectingwhether the fan is operating or not operating, respectively. Althoughillustrated as an external device, projector 1416 can compriseprojection devices (e.g., 320 a-c) on the puck itself as discussedpreviously or on a wand or arm that connects to the puck. In thisembodiment, the projection devices (e.g., 320 a-c or 1416 as shown) arecapable of being directed inward such that projector(s) are pointedtoward the sheet 1420 of vapor. Operational aspects of projectiondevices 320 a-c are discussed previously and the disclosure of theoperation of projection devices is incorporated herein in its entirety.

Fan 1410 is controlled by one or more processors and blows air upwardand outward away from the fan and towards top surface of vaporprojection module 1402. As described previously, transducer 1412atomizes the water and rapidly converts the liquid water into liquidvapor or steam. While transducer 1412 is atomizing the water andgenerating vapor or steam, fan 1410 can be operated to blow the vapor orsteam upward and out of vapor projection module 1402 via the laminarflow tubes 1414 due to the positive pressure exerted by the air flowgenerated by fan 1410.

As described further herein, various other openings may be utilized. Inthe embodiments, where vapor projection module 1402 is partiallycovered, all vapor is blown from the tank 1406 to the laminar flow tubes1414. Thus, at each row of tubes 1414, jets of vapor 1420 are expelledoutward from the upper surface of vapor projection module 1402. Thus,when fan 1406 is operating, a row of vapor jets 1420 is expelled forminga “sheet” of jets.

While fan 1406 is blowing the sheet of vapor jets 1420 through tubes1414, a projector 1416 can be configured to project an image 1418 on thevapor sheet. In the illustrated embodiment, due to the surface area andchange in the refractive index of each droplet of water vapor, aprojected image can be displayed on the sheet of jets 1420. Since thedroplets are likewise transparent or substantially translucent, theresulting projection 1418 appears as a “floating” hologram above puck104.

FIG. 14B illustrates an alternative embodiment of a vapor projectionsystem according to some embodiments of the disclosure.

As illustrated in FIG. 14B, a micro dehumidifier 1404 supplies water totank 1406. Transducer 1412 can be activated (e.g., powered by aprocessing element in a puck) to nebulize the water and rapidly convertthe liquid water into liquid vapor or steam such that the vapor or steambegins to fill chamber 1434. A variable speed fan 1410 blows air intochamber 1434 which projects the vapor upward via positive pressureexerted by the fan's air flow. A sheet of vapor 1420 is expelled upwardfrom the surface of the module wherein an image is projected by aprojector 1416 on the sheet 1420. The operation of the aforementionedcomponents is described more fully in connection with FIG. 14A and thedetailed disclosure of these components is included herein by reference.

Notably, FIG. 14B illustrates the use of a motor-based conveyor belt toselectively choose a row of jets (e.g., row 1426) to activate (providefluid communication to expel vapor) at any given time. As will bedescribed herein, apertures 1422 a, 1422 b act as a “mask” wherein vaporis selectively emitted from the surface of vapor projection module 1402only on a row of tubes aligned with the apertured mask.

As illustrated in FIG. 14B, a conveyer belt assembly includes a motor1430 and a roller 1424 that holds a belt 1428. Belt 1428 can becomprised of a low friction flexible material such as for example PTFE,as can belt guides to guide the belt as it traverses the interior of thepuck. FIG. 14B illustrates an encoder 1432 which can be utilized tomonitor the position of the belt and in turn identify which row of vaporjets is active at a given moment. In some embodiments, encoder 1432transmits this information to a processing device within a puck tocoordinate the display of images on the sheet of vapor as discussedpreviously. FIG. 14C illustrates a belt 1428 with multiple apertures(e.g., 1422 a) which may be utilized to project multiple streams ofvapor simultaneously.

On belt 1428 are at least two apertures 1422 a, 1422 b. In theillustrated embodiment, the width of apertures 1422 a, 1422 b are sizedto match the width of each of the laminar flow tubes such as the row oftubes 1426. As discussed in connection with FIG. 14A (and illustrated inmore detail in FIGS. 14D and 14E), vapor projection module 1402 caninclude a grid of identically sized flow tubes that provide amatrix-like outlet for vapor jets as described herein. As illustrated,the belt 1428 includes two apertures on opposite sides of belt 1428. Bypositioning the apertures 1422 a-b in this manner, one aperture 1422 aor 1422 b is positioned below a tube at any given moment.

During operation, tank 1406 is filled (either partially or fully) frommicro dehumidifier 1404. Once the tank 1406 is suitably filled, a userof a device (e.g., a puck) may initiate operation of the vaporprojection module 1402. In some embodiments, the vapor projection module1402 may be started programmatically. Once initiated, the transducer1412 is activated and begins nebulizing the water to generate steam orvapor within chamber 1434. Simultaneously, the motor 1430 begins torotate, thus moving the apertures 1422 a, 1422 b across the tubes. Whilemoving, encoder 1432 notes the belt position and records the active rowof tubes and transmits this information to a processing device. As theapertures 1422 a-b move across the tubes, fan 1410 is started and airfills chamber 1434 and exert positive pressure on the steam in chamber1434. The positive pressure causes the vapor/steam to be pushed outwardaway from transducer 1412 and tank 1406 toward the matrix of tubes. Dueto the aperture 1422 a, the expelled vapor is forced to row of tubes1426 and exerted as a single row of vapor, thus forming a vapor sheet(as depicted in FIG. 13F). Since the belt 1428 is in motion, the sheetsof vapor are initially expelled from the far left side of vaporprojection module 1402 and proceed in a time series of sheets untilreaching the right side of vapor projection module 1402 until restartingfrom the left side again. In some embodiments, belt 1428 can beconfigured to move backward and forward versus in a circular motion. Asillustrated, the process “restarts” when aperture 1422 a reaches therightmost row of tubes. On the next rotation, aperture 1422 b is thusplaced at the left most row of tubes. The process repeats until stoppedby the vapor projection module 1402 (e.g., manually or until tank 1406is empty). In some embodiments, vapor projection module 1402 can beequipped with additional air vents to selectively project air from fan1410 upward and away from the surface of the vapor projection module1402 in order to clear vapor or steam from above the vapor projectionmodule 1402.

FIGS. 14D and 14E illustrate a matrix of laminar tubes used by avapor-based projection puck according to some embodiments of thedisclosure.

As illustrated in FIGS. 14D and 14E, a matrix of tubing 1436 is placedabove a conveyer belt apparatus including belt 1428. In the illustratedembodiment, the conveyer belt apparatus is similar to that described inconnection with FIGS. 14B and 14C. As discussed previously, an aperture1422 a, 1422 b is moved across matrix of tubing 1436.

FIGS. 14D and 14E illustrate a row or slot of active tubes 1438corresponding to the aperture at a given moment. Notably, row 1438 isintended to highlight the active row and does not constitute a physicalstructure. In contrast, the remainder of belt 1428 (the portionexcluding row 1438) acts as a cover to the remainder of the matrix oftubes 1436. As illustrated in FIGS. 14B and D, in one embodiment, thebelt 1428 can be configured to bisect the matrix 1436. In alternativeembodiments, the belt 1428 can be situated beneath the matrix of tubes1436.

Additionally, as illustrated in FIGS. 14D and 14E, the matrix 1436 canbe situated beneath a rim 1440 of the external opening of vaporprojection module 1402. In the illustrated embodiment, the matrix 1436is partially occluded by the rim 1440, forming a letterbox shape.

In another embodiment (not shown) instead of a belt a circular disk witha slot formed therein along a diameter thereof may be caused to rotateover the tube matrix 1436 to create a rotating plane of vapor to beprojected on. A motor can drive the disk via a gear configured along thecircumferential edge of the disk, and a controller can track thelocation of the slot as it rotates to in turn direct the appropriateprojection on the rotating sheet of vapor in like manner to otherembodiments described herein.

For the purposes of this disclosure a module is a software, hardware, orfirmware (or combinations thereof) system, process or functionality, orcomponent thereof, that performs or facilitates the processes, features,and/or functions described herein (with or without human interaction oraugmentation). A module can include sub-modules. Software components ofa module may be stored on a computer readable medium for execution by aprocessor. Modules may be integral to one or more servers, or be loadedand executed by one or more servers. One or more modules may be groupedinto an engine or an application.

For the purposes of this disclosure the term “user”, “subscriber”“consumer” or “customer” should be understood to refer to a user of anapplication or applications as described herein and/or a consumer ofdata supplied by a data provider. By way of example, and not limitation,the term “user” or “subscriber” can refer to a person who receives dataprovided by the data or service provider over the Internet in a browsersession, or can refer to an automated software application whichreceives the data and stores or processes the data.

Those skilled in the art will recognize that the methods and systems ofthe present disclosure may be implemented in many manners and as suchare not to be limited by the foregoing exemplary embodiments andexamples. In other words, functional elements being performed by singleor multiple components, in various combinations of hardware and softwareor firmware, and individual functions, may be distributed among softwareapplications at either the client level or server level or both. In thisregard, any number of the features of the different embodimentsdescribed herein may be combined into single or multiple embodiments,and alternate embodiments having fewer than, or more than, all of thefeatures described herein are possible.

Functionality may also be, in whole or in part, distributed amongmultiple components, in manners now known or to become known. Thus,myriad software/hardware/firmware combinations are possible in achievingthe functions, features, interfaces and preferences described herein.Moreover, the scope of the present disclosure covers conventionallyknown manners for carrying out the described features and functions andinterfaces, as well as those variations and modifications that may bemade to the hardware or software or firmware components described hereinas would be understood by those skilled in the art now and hereafter.

Furthermore, the embodiments of methods presented and described asflowcharts in this disclosure are provided by way of example in order toprovide a more complete understanding of the technology. The disclosedmethods are not limited to the operations and logical flow presentedherein. Alternative embodiments are contemplated in which the order ofthe various operations is altered and in which sub-operations describedas being part of a larger operation are performed independently.

While various embodiments have been described for purposes of thisdisclosure, such embodiments should not be deemed to limit the teachingof this disclosure to those embodiments. Various changes andmodifications may be made to the elements and operations described aboveto obtain a result that remains within the scope of the systems andprocesses described in this disclosure.

What is claimed is:
 1. A system comprising: a wireless charging device;a wearable assembly; a light field capture VR/AR device comprising afirst charging surface and a second connective surface, the firstcharging surface configured to be positioned on the wireless chargingdevice, the second connective surface configured to be communicativelycoupled to the wearable assembly; and a drone device including a networkinterface, the drone device configured to: receive notifications fromthe light field capture VR/AR device, the notification indicating thatthe light field capture VR/AR device is fully charged, remove the lightfield capture VR/AR device from the charging device, and attach thelight field capture VR/AR device to the wearable assembly.